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BOUNDS AND RATES OF CONVERGENCE FOR THE EXTENDED COMPOUND 



ESTIMATION PROBLEM IN THE SEQUENCE CASE 

I. Introduction and Summary . 

A. The problem. 

Let 9 = (Q.^, 9^ y . . o , 6^y .00 ) be a countably infinite vector 

whose components 0 are elements of some finite interval ft of the 

real line . Let 3 = {p (°): ft ) be for some measure u a family of 

u 

known probability density functions with parameter 9, Let be a 

real valued random variable with density p ( ° ) . Suppose the vector 6 

u a ~ 

1 

is unknown and for each i it is desired to estimate 9.. The estimates 

i 

are to be made in sequence and the estimate of 9^ may be based on the 

independent observations X. j = 1, . .. y i. Thus for each 

J 

i = 1, 2, . o . y a non randomized estimator cp^(X^) is sought for 0^, 
where X. is the vector of observations (X_ , ... . X.). It is assumed 
that at each stage of this estimation problem one suffers squared error 

f \2 

loss, so that if cp. is the estimate of 9. a loss of (cp. - 9.) 

7 i i 7 

units is suffered. The risk of an estimator cp. is defined to be the 

expected loss, that is E[(cp.(X.) - 9.) ]. The average risk for n 

1 ^ . 2 

estimations becomes — V E[(cp.(X.) - 9.) ]. One would like to find, 

n i=l 

for specified ft and 3, a decision procedure cp = (cp , cp^ ^ ... ) which, 
on the basis of its average risk for the first n estimations, is in 
some sense optimal for large n. 
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One way in which such a problem could arise is as follows ; suppose 
the Navy wishes to screen all new recruits and to classify them on the 
basis of their "natural aptitudes' 1 to be radar technicians* In an 
attempt to do this, each recruit is given a test whose outcome can be 
represented as a number. Suppose also that "natural aptitude" can be 
represented on a numerical scale * On the basis of prolonged testing and 
evaluation in the past, the Navy has been able to fit a good probability 
distribution model for the outcome of a person's test score given his 
"true" aptitude as a parameter. The Navy now wants to estimate each new 
recruit's aptitude on the basis of his test score. While squared error 
loss is somewhat artificial, it is clear that the more the Navy errs in 
estimating a recruit’s aptitude the greater the loss it suffers, and 
squared error loss is a convenient way to represent this. In this 
example it is also apparent that many decisions will be made, and from 
the Navy's point of view, the average risk incurred is a reasonable basis 
upon which to judge the "optimality" of a decision procedure. In this 
example, then, would be the i"^ recruit’s true aptitude and 

would be his test score. 

In the preceding example it is not unreasonable to assume each 
recruit’s aptitude is independent of all other recruits’ aptitudes. An 
example will now be presented in which it is not unreasonable to suppose 
the 6 ^ ! s would occur in "patterns." Suppose a Navy anti-submarine 
group is on patrol duty to guard against submarine penetration. It is 
necessary, in deciding what type of patrol to carry out, to have an 
estimate of the average sonar detection range. This range will depend 
upon many different factors such as sea temperature and salinity, as 
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well as the sonar equipment involved. Suppose a test is conducted every 
few hours whose results follow reasonably well a known probability dis- 
tribution with the true average detection range as a parameter. In this 
example then, 9 ^ would be the true detection range and the test 

result. One would not expect 0^ and unre l a ted, however, 

as the conditions fixing their value, while changing, are changing more 
or less continuously in time and a high value of 9 ^ would tend to mean 
a high value of 0 ^ as we H* this example, as in the previous one, 

a decision about the true detection range will be made many times, and 
the average risk is a reasonable criterion to use in evaluating a decision 
procedure. 

B. Known results. 

The problem of finding a good estimator is really twofold. First 
some standard of optimality must be established, and secondly a procedure 
must be found which yields good results according to this standard. 

Samuel [ll] has considered the following standard. Fix 0^. Let G n ( * ) 
be the empirical distribution function of That is 

G (x) = — (the number of i such that 6 . < x) . 
n n i — 

Let (0^; i = 1, . .. , n) be mutually Independent identically distributed 
random variables with a priori distribution function G . If we now con- 
sider X. to be an observation of a random variable with the conditional 

l 

density function p- given that 0 = 9 , then the usual Bayes argument 

e i 1 1 

gives (p^Xp = E[© i |X i ] as the estimator achieving the minimum Bayes risk 
R( G^) . Of course this procedure does not apply to the compound estimation 
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problem since G^ is unknown and in any case the 0^ are not obser- 
vations of random variables. Nevertheless Samuel has shown R(G ) is 

n 

an "optimal" standard to use in evaluating a procedure cp in the 
following sense: Let R (cp, 0) denote the average risk for the first 

n decisions incurred by a decision procedure jp against a parameter 
vector Q. Then R(G^) is an "optimal” standard in that if one considers 
only the class of "obvious” procedures [jp : cp^(X^) = cp(X^) 

i = 1, ... y n] then V n R^(jp, J9) > R(G^). In other words if one 
bases his decision about 6^ only on the observation having 6k as a 
parameter and uses the same rule for each i, one can never achieve a 
lower average risk than the number R(G^) . 

Samuel also gives several sufficient conditions on tty {p n : Ge tt } y 

C7 

and <g which ensure that for each fixed 6 

lim (R n (cp, 0) - R ( G n )) 5 0 

n-» 00 

and in several cases she exhibits specific procedures which satisfy the 
above condition. 

Robbins ( 6 ] [7] [8] and Johns [ 3 ] have done work in the related 
empirical Bayes problem (see Chapter III, Section G) and many of the 
decision procedures they derive are also "optimal” in the compound deci- 
sion problem. Extensions of their estimators will be used in later 
sections. 

C. Summary of new results. 

As mentioned in Section B it is first necessary to establish a 
reasonable standard of "optimality” to use in evaluating a particular 
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decision procedure. Many reasons have been advanced in the literature 
for considering the risk E[(qi - 6h) ] to be a good indication of how 
well a particular decision rule does. In the compound decision problem/ 
it seems even more reasonable to consider the average risk R (cp, 0 ) as 
a reliable index to be used in evaluating a particular decision procedure 
(£, and this is the index adopted in this paper. A standard R(_0 n ) is 
now needed such that if for all Q and n R(jj) , ) is no greater than 

R(_0 ), one would be willing to say (£ is a good decision procedure. 

Samuel has given good intuitive reasons for selecting R(0 ) = R(G ), and. 
has made the statement [ 11] that R( G^) cannot, in the limit, be improved 
upon. Based on an idea of Johns [5], a sequence of more stringent stand- 
ards (R^(£ ) : k = 1, 2, ... ) will, however, be obtained in this paper such 
that R^(£ n ) = R(G n ); and for any fixed k = 1 , 2, ... and for all 0, 

R k ( -n ) = + n > + h < k > n > i) where f ( k > n, 0) > 0 and 

h(k, n, 6 ) - 0(~) uniformly in 0. In addition for "most" 0, 
f(k, n, 0) is in fact strictly positive. R^(j0 n ) will be shown to be 
the minimum Bayes risk possible if in fact is a realization of an 

n dimensional random vector whose last k components are independently 
distributed from the first n - k components according to the k 
dimensional empirical distribution function generated by J? . The ex- 
tended compound estimation problem is defined to be the problem of finding 
procedures which asymptotically achieve these standards. The analogous 
problem in the empirical Bayes case is being considered by Barndorff- 
Nielsen [l]. To make these statements more explicit several definitions 
are needed. These definitions will be used throughout the paper. 
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Def. l) Let ft be a bounded interval of the real line. Let 

3 = {p^- Oe ft } be a family of probability density functions with respect 

to some measure p. Let [6 : 0.eft j = 1, 2 y J, ... ) be an arbitrary 

J J 

sequence. Let {X . : j = 1, 2, . . . } be a sequence of mutually tnde- 
J 

pendent real valued random variables with X. distributed according to 

J 

p. . Let X = (X, ; ... , X.). Let 0 = (0 , 0 p , ... , 0 , ... ) where 
j i j 1 -L d n 

0^eD 1=1, 2, ... . Let 0^ he the vector consisting of the first 



n components of 0. 



Def. 2) V it n, V k = 1, ... , n the order empirical dis- 



tribution function of 0 is: 

— n 



G n (y l’ y 2’ ••• ’ V = — k ' V ' l 

(# of j (k < j < n) such that: 0 



j-k+,0 



5 y £ ^ ^ > • • • > k) 



When k = 1 this definition yields the usual empirical distribution 
function. 

Let k and m be fixed arbitrary positive integers k < m. Let 

{©^: i = 1, ... } m} be a sequence of random variables with range space 

ft. Let 0 ........ 0 have an a priori joint distribution function 

G and assume the remaining are distributed independently of 0^. 

Let {X^: i = 1, ... y m) be a sequence of random variables with 

conditional density functions p^ given ©^ = 6^ such that the X^ 

i 

are mutually conditionally independent given the 0^. For estimating 

the realization 0 of 0 it is well known that the estimate 
m m 

E[© |X . ... . X ], which depends only on the last k observa- 
nt m-k+1* ’ m ’ 

tions, is a Bayes estimate and achieves the Bayes risk B(G). 
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Def o 3) V 0, n, V k = 1> * . . , n 

let R^(_0^) = R(G^) where G^ is the k^ n order empirical distribution 
function of 0 . Thus R. (0 ) is the Bayes risk for G^. 

Using the above definitions it will be shown in theorem l) that 

f(k, n, e) = E{[E(e^ +1 |x k+1 ) - E(© k+1 |x 2 , ... , \ +1 )] 2 ) 

k + 1 

where 0. . ... . 0_ have the a priori joint distribution function G 
1 k+1 n 

It is clear that f^ is always non negative and will equal zero only if 

E ^ 0 k+1 J^k+l-' = ^k+l^' •** * "l+l ^ with probability one. This 
condition is clearly satisfied for most 3 only if 0^ generates an 

k+i 

empirical distribution function G such that 0. and 0, are 
independently distributed* It is not unreasonable to suppose that "few” 
arbitrary sequences, occurring in situations leading to the compound 
decision problem, will satisfy this condition, even as n approaches 00 . 
Another necessary condition for E[0^ [X^^] = E[0 |X , 00 . , X^ + ^] 

is that the sample serial correlation coefficient lag k+1 of 
{0^: i = 1, , n) be zero* Again it seems unlikely that many 

sequences of 0^ would have this property, especially for small values 
of k* In particular if 0 has repeated ,T patterns ,T of length greater 
than k, neither of these conditions would be expected to hold* 

Accepting R^(0 n ) as a standard to be used in evaluating a decision 
procedure cp, attention is turned to constructing procedures for specific 
classes 3, and to evaluating these procedures* 
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Def • 4) Let £ k (^, 0) = R^cp, 0) - 1^(0^. Thus 0) 

represents the difference after n decisions between the average risk 
attained by a particular decision procedure and the k^* 1 standard. 

For many important classes 3, including the normal, gamma, a 
discrete exponential family, and a "non- parametric" class, decision 
procedures _cp k will be found and an upper bound B(k, n) will be given 
such that £ k (<£ k , 0) < B(k, n) for all 9e £2°° and such that 
lim B(k, n) = 0. For the discrete exponential family, which includes 

n->oo 

the geometric, negative binomial, and Poisson families, it will be 

k 

These results represent a considerable 



shown that B(k, n) = 0 



log n 

1 W 



n 



improvement over those obtained by Samuel [11] who considered only the 
case k = 1 and showed 



lim TR (cp, 0) - R, (0 )] < 
■ n -x-> l'-n' - 



n-» oo 



for any fixed 0 in a parameter space more restricted than that con- 
sidered in this paper. If 5 is the class of binomial probability 
density functions, a decision procedure is obtained which attains a 

J log n 



lower average risk than previously known procedures, and 0 l”p 74 

in' 

obtained as the rate of convergence of this risk to its "standard." 



is 
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II © Preliminary Results o 



In this chapter we shall first prove that R^(© n ) = + 

f(k, n, 9) + h(k^ n* 9) where f and h have the properties stated 
in Chapter L We shall then develop a general theorem and corollary 
which will enable us to obtain specific decision procedures in Chapter III© 
Finally we shall prove several lemmas which will be useful in Chapter III. 

Del© 5) Let v = (y, ? Y n s> * • * > y ) be an arbitrary vector© 

l d n 



While both Q and Q* are functions of several variables which are 
not explicit in the notation, it will always be clear in context what 
arguments are intended. We note that Q (x^) is the unconditional 



are assumed to be random variables 0^, ... , 0^ with a priori distri- 
k 

but ion function G . 




Def. 6) y 3 , y k, n such that 1 < k < n let 







n 
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Def. 7) V 3 , 0, V k, n such that i < k < n, let 




■ \ 




if 



Q. 



n 



> 0 



0 



otherwise 






Let m and n he integers such that 1 < k < m, and n < °o, then 

ilf^(X^) is one version of E[© [X^], and R. (0 ) = E[(© - \|r k (X k ) 2 ] 
n — nr 1 m 1 — m ’ k — n m n — m 

= E[0 2 - (^(X^) 2 ]. 
m T n — m 

Def. 8) V ^ j such that 1 < k < j let 



f ,[< p, e k ] = e[(<p(A - 0 ) 2 ] 

<J J J w 



where cp( • ) is an arbitrary non randomized estimator with a 
k-dimensional argument. 



V n > k let R(<p, 9 n ) = —FT I .1 F j [ ^ -?j ] • 

J~ K 

R(cp, jQ ) is then the Bayes risk in using the rule cp(X k ) as an estimate 
of 0^ when © 1 , ... , © k have the a priori distribut or! (1. 

We now compare R^(0 ) with and prove: 

Theorem l) y S , 0, k, n 1 < k < n < °° \(® n ) = \+] ^-n^ ' n ’ + 

h(k, n, 0) Where f(k, n, 0) > 0 and |h(k, n, 0) | = oj-ij uniformly 

in 9. 
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Proof : 



Let E^[*] refer to expectation vith respect to and E^[*] 

refer to expectation vith respect to Eor an y estimator 

^W' V = V^W - e k + i )2] 

= E 2^®k+l' " E 2^ E 2^®k+l^+l^ 

+ ¥ [E 2 «WW - ” ( 4 + i) l2 > 

which of course is minimized for cp(X^ + p = E 2^®k+] k+1^ ‘ Getting 

' p( \ + i ) ■ V'W^+i 1 we oW - ain 

¥ E n> - WV - V¥ - R <¥‘Wl&i ) ' V 

+ V'WJW - Wl^l” 2 ’ • 

Let f(k, n, 9) = E 2 ([E 2 (^ +1 l4 +1 ) - E 2 (^ 1 |X^ +1 )] 2 } 

h(k, n, 9) = yej - R(E 2 [e k+1 |x^ +1 ], e n ) 

Then it remains only to show |h(k, n, 6) | = oj-^J uniformly in _0. But 



since 



W = E i [e k ] - E i( E i [e kl2k'l 



and 



R ^ E 2*- 0 k+il-k+l^ -n^ E 2*- 0 k+l-* " E 2^ E 2^ 0 k+l ^-k+1^ 
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and since fi is a bounded interval, say 0eft=J*|0|<B<«> we have: 



|h(k, n,e)l< - E 2 (^ +1 )| + |E 1 (E2[e k |X k ]) - V^e^lX^]) I 



and 



IV# 



e 2 K + i>I 



1 n o i n 

1 v _ 1 

n - k + 1 J-i ,1 n - k , rt 



j=k 



J=k+1 J 



(n - k + l)(n - k) 



n 



I e; + (n - k)e: 
j=k+l J 



< 2 

- n - k + 1 



Also 



- E 2 (E l [e k + iii +1 )J i 




\ 2 n 



ft 



(xj 



J=k 1=1 .j-k+l 
n - k + 1 



/ n k \ 

i e j n p 0 y z ) \ 

j=k+l J 2=1 i-k+2 


n k ' 

i fl p e ( x P 

j=k+l 2=1 j-k +2 


) ^(dXj^) 


\ ^ ft p e /V i 

\ j=k+l 2=1 j-k+i / 


n - k 


/ 





For fixed x^ the expression inside the braces may be written as 



1 a + a | 


, b + b 


a i 


i b 


n 


n 


_n 


n 


b + b 


n - k + 1 


b 


n - k 


• n 1 


f 


nJ 


1 
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where 



a n I e j II P 0 (x i) 

J=k+1 3 4=1 j-k+4 



a ■ e * k 



b n = I n Pe ( X P 

j=k+l 4=1 j-k+i 



b ■ k \ u ‘ } 



and 



n 



n 



b + b 
n 


a 

n 


2 


b 

n 


n - k + 1 


b 

n 


1 n 


- k 


> (a + a) 2 - 
n n 


(n 


- k 


+ 1) 



^ + D )a 

n n 



n. n 



! (n - k)(2b a a + b a 2 - b a 2 ) - b a 2 - b a 2 
1 n n n n n n n 



(n - k)(n - k + l)(b n + 



- n ~ k + 1 



2a a 



n 



b + bl 
n 



b + b 
n 



b a 



n 



(b + b)b 
v n n 



(n - kj(n - k + l) 



n 



b a 



n 



b + b 7b + b)b 
n I r n 7 n 



From the definition of a , b . a, and b it is clear that 

n 7 n 7 7 



a 

i- r 



- < B — < B b > 0 b>0 so that 

'b 1 — n — — 

n 



2a a 

n 


< 


2a a 

n 


_ Q 


a | 


a I 


■h 


b + b 
n 


b 

n 


— e± 


b 

n| 


b 


u 



< 2B 2 b 
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2 




2 V 


a 


< 


a b 


b + b 


.2 


n 




b 



2 

< Bb 



b a 



n 



(b + b)(b ) 
n n 



< 



b a 



n 



2 

< Bb 



2 

a 

n 


< 


a 2 b 
n n 


b + b 


,2 


n 




b 






n 



Thus we have 



- E 2 (E 2 [e k+ll^+l 1, l 

. n hB 2 b B ^ b n 1 b) I , 

-Jjn-k + 1^ (n - k)(n - k + 

R k 

■ n -‘‘k * 1 / ( 

_k 




4b £ 

n - k + 1 ( n - k) ( n - k + 1) n - k + 1 
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Hence 



|h(k, n, 0)| < 



7B 



n - k + l ' 



The implications of theorem l) were discussed in Chapter I. 
state and prove a generalized form of a lemma of Samuel [11]. 

Lemma l) y 0, k > 1, n > k 



Q.E.D. 

We now 



^—r~T I E[(^ k (X k ) - 6.) 2 ] < R ( 0 ) 

n - k + 1 .A. T i — l \ — k — n' 



i=k 



,k „kx 



Proof: Fix 9, k, and n. Using the expressions F (\|r., 0.) and 

J + j 

R(\(r k , 0 ) given in definition 8), and observing that R(t k + ^> ®j_) > R^j^) 
we have: 



n-iUl J, 



i=k 



rvr T I i F -<* k - \ y -(* k - ° k )r 

n - k + 1 lk i A j 1 _j A J 1 _j 1 

= --T— A { (I " k + l)B( *i’ -i’ " (1 ■ k>H( ^’ -l- E> } 

= i I (i - k + l)[R(^ k , e.)-R(i)/ k , 0 . ) ] + R(\Jf k , 9 ) 

n - k + 1 .A. l —l l+l —l n — n 

i=k 

< R(t k , 0 ) = R. (0 ). 

- Y n — n k — n 

Q • E « D • 
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Def. 9) 



qp k (x ), 

n— n ' 



k. ic h 

A decision procedure cp = (cp^(X^), cp^X^), ... 
is asymptotically optimal of order if 



> 



V 0 



lim 
n-> oo 



1 

n 



X E [(^ - £>/] - yep 



< 0 . 



r . n 

If lim j sup ^ £ 

n-> oo ^ q i=I 

"bll 

uniformly asymptotically optimal of k order. 



E[(q>: 



^i) 2 ] 



R (0 ) 

k — n 



< 0 



then 



2 



is 



We shall now state and prove a general theorem, which with its corollary 
will enable us to obtain the results of Chapter III* 



Theorem 2) V bounded interval ft = [a, f3], family of densities 3, and 

integer k > 1, let cp be a decision procedure such that y i > k 

P[cp.(X.)cft] = 1* Suppose there exist non negative functions £.(0, x,), 
1 ■“ 1 1 *“K. 

£^(0, x^), and a j_^— ) such that y 0, x^> i > k 

a) Pfjcp^xp - ^(xpl > i.(e, = xj < q(a, 

b) lim ji Y 2^) + ^(0. 22 k )lQ i (22 k ) 



n-» 00 i=k 



+ i I PCQjxJ) < a.(e)3 



i=k 



= 0 



uniformly in 0 



where the functions and are as given by definitions 6) and 

7 ). 

k 

Then the decision procedure cp is uniformly asymptotically optimal 
of k^* 1 order, and moreover 
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eV, 0) - ■ * 2lg_L-gl £ EU . + (p . a)? . | Q . > a .] 

n- 1 — — n n s i i' 1 — 1 

i 2(p - a) : 



rx 

I 

i=k 
>2 n 



E PfQi < a.] 



i=k 



,k, k 



where ^(cp , 0) is given by definition 4). 



k k. 

Proof: Clearly it is sufficient to prove the upper hound for £ n ( eg , G) 

is correct c 

We first represent as a sum of several functions, and then examine 

each of these functions. We shall consider k and n fixed. 

Let : 



k- p 

H l(cp k , 0) = i £ (E[(cp k - e.) 2 ] - 1^(0)]} 
i=l 



H 2 (cp k , e) = ^ I {E[(cp k - e.) 2 ] - E[(| k - e.) 2 ]} 
i=k 



H,(cp k , 2) = i I (E[(+i - e i) 2 ] - 

^ i=k 



For the remainder of the proof we delete the superscript k. Clearly 

£ (cp, 9) = — Y E[(<p. - e.) 2 ] - R ( 0 ) 
n-*’ n ,'H i' k — n 

i=l 

= H^cp, 0) + H 2 (qp, 0) + H ($, 0) . 

Let B = ( p - Qt) . Then E[(cp. - 0.) 2 ] < B 2 and R, (0 ) < B 2 ; hence 

2 l l — k — n — 

H^(cp, _©) < ^ k ~ . It follows immediately from Lemma l) that 

H^(cp, _0) < 0. It remains only to examine H 2 (cp, 0). 
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1 n 

0) = - I E[(qp. - t i )(<p i + ♦ i - 2© i )] 



i=k 



<^f I E[|<p. - * |] 
i=k 



< ^ I min [E[|| + B£ |], B] 

i=k 

<— I E[| + BC |Q > a ]P[Q > a ] + ^_ £ PfQ. < a,] 

i=k n i=k 1 



The desired result follows immediately. 

Q.E.D 

Corollary. If condition b) of theorem 2) is replaced by 
b') lim E[|.(0, xj) + ^.(0, xj)] = 0 

i— » oo 

uniformly in 6 

k t h 

then cp is uniformly asymptotically optimal in the k order. 

Proof: From the proof of theorem 2) it is enough to show 

lim {sup H (cp\ ©)]• < 0, recalling that H (<£, d) is a function of n 
n-> 6 
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But H 2 (cp, Q) l E[|cp. - * |] 

i=k 

< — T EU.(e, x k ) + B^.rf)) 

— n 1 — 7 — 1 —v 



n 



-> 0 uniformly in 0 as n -» «> since uniform convergence 
implies uniform convergence in Cesaro mean. 

Q • E . D . 

We turn now to several lemmas which will he useful in Chapter III. 
lemmas 2), j), and 4) will be used to establish condition a) of theorem 2), 
while lemma 5) will be used in evaluating certain limits. 

The first of these is an inequality proved by Hoeffding [ 2 ], which 
we state here without proof. 



Lemma 2) If X , ... , are independent and a < < b 

for i- = 1, ... , n then V t > 0 

2 



P[ I X - E[X] I > t] < 2e 



Hb^l 



i n 

where X = - Y X. . 

n i 

i=l 
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Lemma 3) Let X. , X~, . . • . X 0 <X.<b i = 1, . . • , n be 

2* ' n — 1 — 

a sequence of random variables such that for some k > 0 and 
V i = 1> 2, . .. , k the random variables X^, X^ + ^j ^i+2k* 600 are 
mutually independent . Then 



P[ |x - E[X] i > 6] < 2ke 



P n c-2 
^k(n+k) 



m 



Proofs Let 


S . = V X .. , . where m 
1 j=o Jk+1 


is defined as the 


integer 


such 


that r - 1 
k 


< m < and X„ - 0 for 

- k £ 


H > n. Let y . = 
' 1 


E[S i ], 


Let 



A. = event is. - 7 . 1 > But S. is the sum of m + 1 independent 

1 1 1 ' i 1 - k 1 

random variables and from lemma 2 ) we have: 



P[A ] = P 



m 

I X 
j=0 



i \ Sri 
jk+i — k_ 



m 



= P 



m + ^ - k(m + l) 



- 2 (m+l) 



c-2 2 

o n 



< 2e 



.2 ( .-> 2,2 
k (m-t-1) b 



-2 



< 2e 



2 a 2 

n o 
k(n+k) b 2 



20 



So that: 



P[|X - 



E[X]| >51 = pf| l (S - 7 )| >6n 
l i=l 

< Pfl |S i - rj > 5 : 



» 1 - P 



[ IjSi - rj < 6n 



< x - p [i2i a;. 



= P 



k 

U A 
li=l 



J < E PU ] 

J i=l 



-P- 



2 * 2 

n o 



< 



2ke k(n+k H 2 



Q.E .Do 



Lemma 4) Suppose for non negative random variables X^ , 
p il x i - nj > 6^] 5 € i i = 1, 2; > 0, M 2 > 0; and B ^ 0 is any 

^ 1 X i 

number such that — < B < Define W = min[rjr-^ Bj (if X = X = 0 
we take W = 0) . Then 



P[|W - 





< 5 1 + B6 2» S'l*' 2 
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Proof: We first show 



l«-^l (I*! - t^l +B|x 2 -ig) 



2 2 



Case i) W < B. If X g = 0 then X ± = 0 and 



|W '^' ’ l ^ l5 ^ + B "i (|X l' ,I l l +B l X 2- lJ 2 D 



If X Jo then 



|w - J ' 'xj - J * =grl< x i - d i > x 2 - < x 2 - d 2 > x il 
- ^ ( I X 1 ' “il + B l X 2 • B 2 I) 



X, 



Case ii) W = B. Let Y = — . Then Y 4- 0 and Y > X, 

-D — 



<;r (I^L - bJ + b(i> 2 - x 2 ) <^- (lx, - Hjl + b|x. 



1*2 1 ) 



From this fact we have 



l x i - M il < 5 i l x 2 - b 2 I < & 2 => l» - JtI <£ 1 >« 2 ) 



22 



and 




B5 2 )] = 1 - P[|W - J^| <i (5 1 + B5 2 ) 

< 1 - P[|X 1 - nj < 8 1 and |X 2 - n g | < 5^ 
= P[|Xi - pj > 8 1 or |X 2 - n 2 | > 8 2 ] 



Q * E • D • 



Lemma 5) Let F be an absolutely continuous distribution 
function with corresponding density function f. Let 
= (x: |f"(x)| <M and f"(x) is continuous). Let 
D = (x: | f ' (x) | > 0) . Let ye( -°° , 00 ) - 

i) If there exist M < » e >0 such that: yeD and 

' o 

(x: y<x<y + e } c C.. then V e 0 < e < e 
J J o M — o 



ii) If there exist M < °° e Q > 0 such that: yeD and 

(x: y - e Q < x < y) c C M then V e 0 < e < e^. 



^ J f (x)dx = f (y + pe) with |p - || < g| f ^ y )| e 



y+e 



y 




f(x)dx = f(y - Pe) with |p - -| < ? | € 
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Proof: Using the mean value theorem and the Taylor expansion we have 

in case i) 



i J f(x)dx = f(y + pe) = f(y) + f'(y)pe + pe)^ 

y 

where 0 < p < 1 and y < x* < y + e • 

But: 



y+e 3 » 

i /*,)«, -±(f<y+«)- F(y)) -i f(y)c + iddc 2 + £ifd^] 



where y < x' < y + e . 
Thus 



f '(y)pe + . £hh + 



(p - f)f'(y) = 



f"(x') _ f"(x*) p p| e 



=» |p- |llf(y)l < + 



1 


M . M 


|f'(y)l 


|5 + 2 j 






e = 



2Me 



3 | f ' (y)T 



The proof in case ii) is similar. 



Q.E.D. 
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III. Main results. 



We now turn to the task of finding asymptotically optimal procedures, 
bounds, and rates of convergence for specific classes of distributions. 

We shall also look at a modification of our problem in a very general 
class of distributions. Finally we shall consider the "empirical Bayes” 
problem. 

The notation we shall develop and use is inherently cumbersome; to 
ease its burden somewhat we shall not always indicate all possible 
dependencies and shall not always indicate one or more of the arguments 
of a function. Hopefully no misunderstanding will arise because of this 
practice . 

A. A special discrete class of distributions. 

We first consider a special discrete class of distributions defined 
on the non negative integers as follows: 



p[x = xje] = p e (x) = e x h(e)g(x) x = o, i, 2 , ... 
where: i) Qz Q. = [0, P] 0 < P < 00 h(P) > 0; 



ii) There exists M* < °° such that: )/ 9e Q E 




iii) There exists M' < °° such that V J = 0, 1, 2 } 
such that g(i) g(j) / 0 
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iv) If p < 1 then there exists a constant b such that 

g(x) < x for all but finitely many integers x. 

If (3 > 1 then there exists a constant b such that 

g(x) < for all but finitely many integers x. 

x 

All of these restrictions are quite mild. The third prevents g from 
oscillating wildly as its argument progresses through the integers. The 
second and fourth conditions restrict slightly the rate at which 
0 X g(x) -» 0 as x->°°. 

Examples of such a class are: 



Type 


e 


h(e) 


sW 


Poisson 


[o, < » 


-e 

e 


1 

xl 


geometric 


[o, p],p < 1 


(i - e) 


1 


negative 

binomial 


[0, p],p < 1 


(l - e) a . a > 0 
a assumed known 


fa + x - 1 

l x i 



The conditions are easily seen to be satisfied. 

Recalling that x^ = (x , . , ... , x.) j = k, k + 1, 

J J—Ki" _L J 



1 K / = 



we define : 



* 






J = k, k + 1, 



0 otherwise 



P i^ “ i - k + 1 jf k 



i = k, k + 1, • • • 
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( x i > ••• > ^ -I y + l) 

— Vi)p t l^ if + > 0 



p ± ( i£k> " < 



0 



otherwise 



i = k, k + 1, 



if i = 1, . . . , k - 1 

if 0 < P*(xJ) < P i = k, k + 1, ... , 
if P < P*(X^) i = k, k + 1, . . . , 

Theorem 3) For the problem defined in this section 

T k = (qf, q>£, ... ) is uniformly asymptotically optimal of k tn 
1 d 

order for k = 1, 2 , • •• , and < B(k, n) = . 

Proof: We shall show the conditions of Theorem 2) are satisfied. 

Fix k. Fix 0. 

Recall Q.(xJ = T~TTT~T £ JI p e ( x t) i = K k + 1, ••• 

1 ^ 1 k 1 j=k /=1 j-k +i 1 

We observe that there exists a set in k dimensional space such 

that PfX^eR..] = 1 and x^,eR^ g(x^)Q^(Xj c ) >0. We then have 

x^eR^ =» g(x^ + l)Q i (x k ) > 0. yx k eR i we know from definition 7) 

that y i > k: 



*ta> 
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<tek> 



>k 0J Hi Vk+A^j-k+^V 



I Jl 

j=k i=l 

g (x k )Q i (xi, x 2 



8 j-k+i h(e j-k + P g(x /> 



• • > \-r \ + ^ 

s( \ + ^ w 



We now consider the Y 's. 

J 

independent it is clear that 
Y £'^k h f it-k^k^ Y i+2k ; £k'’ 



Since the X 's j = 1, , i are 

J 

} i = 1, 2, . « e j the random variables 
are mutually conditionally independent 



given 




x, ♦ Also : 

-K 



2^=2*) 

= n PA ( x /») j = k » > 1 - k 

jf=L j-k+i * 

0 < E[Y ( 3 ^) |xj = < 1 j = i - k + 1, . . . , i 

kg(x + l) 

Now v x i t eR i V e ± ) > ~ k Vi' 1 = 2k ' ■“ 

we have, using lemma 3) : 
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PtlgC^ + l)P i (x k ) - gCxj^ + l)Q 1 (x k )| > Sjx^ > 3^3 

I p i-k^k^ 



< P 



Q i-k^k^ + U - 2k + 1 j =i ^ k+1 Y J^k^ 



(xj| 



i - 2k + 1 . . ^ A 6* . t / i 
J=i-k+l /=! j-k+i 



8 i (i - k + l) 



- g(x fc + l) (i - 2k + l)l-i -^k_ 






< P 



B.(i - k + 1) 

l P i-k^k^ " Q i-k^k^ - g^ + l)(i - 2k + 1) 



" (i - 2k + l)l^i _ *k 



< 2k exp {- 2 k j* I f/pp 2 . 



5.(i - k + l) 



i ’ ' k 

.gCx^ + l)(i - 2k + l) " (i - 2k + 1) 



< 2k exp i —r 



kb. 257 ( i - k + 1) 

1 1 



gu k + 1) 



k S ( X k * 1 ) 



kgCx^.) 

By a similar argument V x^h^ V e^x^ 6^) >— ^ — we have: 



P[|g(x k )P i (x 1 , , .x^, \ + l) " s(x k )Q i (x 1 , . .. , x k _ 1 , x k + 1)| 



r 4e 2e 2 ( i - k + 1) 

> € . jx. - x, ] < 2k exp i—7 r - ~o~~ v — ~~ 

- i'-i -k - * le(x k j k / (k k) 
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We are now in a position to apply lemma 4) to obtain the functions 
^ and £ for condition a) of theorem 2). If we substitute in 
lemma 4) 



W = cp^(X ± ) 



Wj = ••• j \_ x > \ + i) 

g 2 = s (\ + :i -)Q 1 (x k ) 



Then we have V i = 2k, 2k + 1, . . . 




e . 



l 



+ P6 



h ( ^ )g(x k + 11 




< 2k 



2e'. ( i-k+l) 

kg 2 ^) 



4e. 25 2 ( i-k+l) 

i i v _ 

g ^ X k' ) kg 2 (x,+l) 

, K 



46, 



g(V x) 



Recall the above inequality has been shown only for Xp e ) i > 2k, 
kg(x + 1) kg(xj 

6 i >_ ... j- , e >_ — H ^ . if any of these conditions should not 

hold we shall use the trivial inequality 



Let r S . = 



€ . + p 5. 

11 



1 1 ( -l g(X k + U 



he . US . 

i , = 2k exp + -T-— ~TT 

1 [g'^v si\ + i) 



2(i - k + 1) 
k 



2 
€ . 
1 



5 2 

1 



g 2 (x k ) g 2 ( x k + !) 



We have now produced the inequalities for condition a) of theorem 2) « 
We must now choose functions subject to the 



conditions that 



k + l) 



D i < i - k + 1 



5. = 0 

1 



e . < - 






i i - k + 1 



€. = 0 
1 



and show that for Q. and some a. condition h) of theorem 2) is 
satisfied. 

We first prove the following lemma. 

Lemma 6) For {p~: 6e ft) defined in this section V k = 1* 2* ... 
Vs 0 < e < 1 there exists a constant M = M(g <? k) <00 such that 
V Oe n °°, v n = k, k + 1, ♦ . * } n / 1 



n r i k 



l p 

i=k 



x n „ 

L j=k i=l j-k+i 



. ,»««> < 1 



< Mn £ log k n 
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Proof; We first consider the case k = 1. Let d be the smallest 
integer such that g(d) ^ 0. V m = d, d + 1, . . • , V 



y p_(x) 

L - Q 00 . 

x=m _ y @ x-m g(x) 

L gtmJ 



p 0 (m) 



x=m 






+ V 0 y ^.y. + . m ) 

y=0 



- W) 






where M" < oo from condition iii) and the 



easily verified fact that h is a decreasing function. 



Also it is clear that y 6 there exists a smallest non negative integer 
i 

m. such that Y (m.) < i . 
i 9 . l 

J=1 J 



Let 



I(a, b) = / 



1 if a < b 



0 otherwise 



Then : 



n 

I p 

i=l 



I P 0 (X.) < i 
J=1 J 



. € 



n oo 

= I I 1 

i=l x=0 



Z P e (x), i 
j=l j 



. e 



P 0 (x) 
1 



n oo 

< i i p 0 w 

i=l x=m. i 



M ,f 



n 



±W 1 X. P 9. (m i ) 
1=1 1 
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i i 

Nov Y, P q ( x ) - Y 0 ^( 9 i)g( x ) < so that If 

J=1 J J»1 J J 

x 

P h(0)g(x) < i e then < x. But p X h(o)g(x) > i e-1 

=4- x log P + log g(x) > (e - l) log i - log h(o) 

If P < 1 then ve have from condition iv) that 

x log p + h log x > (e - l) log i - log h(0) 



==> x < 




log i + 



log h(0) 
1 - e 



+ b I°g x 
1 - e 



=» there exists M" 1 < °° such that x < M" 1 log i for i > 1 . 
If P > 1 then we have from condition iv) that 



x log P - (x - b) log x > (e - l) log i - log h(0) 



x(log x - log P) < (l - e) log i + log h(o) + b log x 



But x > P log x - log p > log P 



so if x > p 2 ^ 1 then also x < -i 

T - log P 



ice i + issJM + in 

° 1 - e 1 



If p = 1 then if x > 1 also x < 

In each case there exists M 1 " < 00 such that x < M"' log i 

for i > 1. 



. , leg h(0)' , 

log 1 + \ * 



log X 

- £ 



b log x 
1 - € 
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Let M., be the smallest integer such that M. > M"' log i for 
M 1 

i > 1. Then P h(0)g(M^) < i € and hence m^ < M^. Nov let 
I = {i: i = 1, 2, ... , n and m^ = v} for v = 1 , 2 , . .. , M^. 
Let i be the greatest integer in 1^. Then 



I 

i=l 



I 

L j=i 



P 9. (X i> 



< i 



M" 



n 



- h(p ) £ p 0 / m i) 

i=l i 



M" 

= W 1 




p 0. (v) 



< M " 

-W) 



M 

n 



I 



V=1 




from the definition of m. 

l 



M" M 

. n e . e .. 

< n < M n log n . 



This completes the proof for the case k = 1. We now consider the case 
k > 2. 



n 

£ p 

i=k 



£ ft 



i 

j=k J=a w j-k+i 



'9 



< X i-k + i> < 1 



. G 



m.-l 



r n «> oo i 

y y ••• Eli 

L i = k v° Vr° *k =0 



ft 



I 

L j=k i=l J-k+i 



(x,), r 



p 0 ( x p) 

t = 1 i-k+i - 



n oo 



00 oo 



I I 

c l" 



L i=k x, =0 x^-^O X k =m -‘ 



I I p 
1 



i n p e ( x i)> 1 

j=k i=l j-k+£ 



. G 



p fl ( x p) 

£=1 i-k+£ 
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where m. is as defined for the case k = 1. We shall now consider the 

i 

two bracketed parts of the right hand side of the last equation separately. 
The second bracketed expression is clearly seen to be less than or equal 
to M*n G log n for some M* < <» by the argument used in the case k = 1. 
The first bracketed expression can now be broken up into k expressions, 
k - 1 of which are less than or equal to M*n^ log n with the remaining 
expression being 



m.-l 
n i 

I I 

i=k x^O 



m.-l 

l 

I i 

v 0 



1 TT 

I II Pe Up, 1 

j=k i=l j-k+i 



. £ 



ft 



Pfl up 

=1 l-k+i £ 



But 



for each (k-l)-tuple of possible values (x^, x^, 



either I 



1 FI % 

■ j=k i=l j-k+i 

x^ = 0, 1, . . . , nu - 1 or there exists a non negative integer 

a^ i^ X l* ^ 5 nii - 1 such that the indicator is zero for 

x^ = 0, 1, ... , ^ - 1, and one for x^. = a^ ^ . We may now use the 

same arguments as in the case k = 1, for those (k-l) -tuples 
(x^, ... , x^ p such that the indicator function is not zero for 

x k = 0, 1, . . . , nu - 1, to show that 



< \-i> 



UP, i € 



is zero for all 



v, m.-l . . 

n i r l k 

i 1 1 1 1 II 



i=l x fc =0 L j=k i=l j-k+i 



up, i 



ft 



p e Up 

i=l 1-k+g 



n oo k 

< i in p 0 ,<*<> 

- i ' 1 V a k,i i=1 1 - k+l 
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Thus : 




m.-l m.-l 



n 1 

Z I 

i=k x =0 



Z I 

v° 



i k 



i n p 0 (*<>)> i e n 



M 

n n 



b'Q 

j=k i=l j-k+i 

M m.-l 

n 1 



Pfl (x.) 
i=l i-k +£ 



< Z Z ••• Z Z I 

i=k x 1= 0 x k _ 1= 0 x k =0 



i k 



z n p e i e n 



<9 v ^ 

L j=k i=i j-k+i 



Pa (x.) 
i=l l-k+i 



M n M n M" M n € 

< Z - Z 1^- 

x l=° *k-l“° 



M"(M ) k n e 

< A 



Upon combining the above result with the previous k inequalities we 
have the desired conclusion, and the proof of the lemma is complete* 
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We turn now to the task of showing condition b) of theorem 2) is 

satisfied for a suitable choice of g . and t . . The theorem will 

1 *1 

k k 

then give us an upper bound for y 6) and we shall then see it 

has the claimed rate of convergence to zero. Recalling 



% i - k + 1 £ ft p 0 / X i-k+^ 1 k, k + 1, , n 

j=k i=l j-k +JI 



and 



l/l*. 1 n M lo k n 

defining a. = [-] we have from lemma S) - 7 P[Q, < a.] < rfj- — 

1 ' 11 n i=k n' 



i=k 
e. + P&, 



Earlier 



in the proof we shoved we could take |^ = q g(x,' + l) 



and 



= 2k 




2e^(i-k+l) 

kg 2 (X i ) 



+ k- 



g(xT) 



25^( j-k+l) if5 i 
kg 2 (Xj+i) g(x k +1) - 



for arbitrary functions and such that 



kg(X + 1) 

V 048 i-> 'i-'Wi- 



> 



Xg(X^) 



i - i - k + 1 



Let 



i be the smallest integer such that 

o 

and such that i > 2k. 

o — 



log *0 

7T 

o 




o 



k 

k + 1 
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Also let 



g(X + 1)Q log i , 

if > ■“T7I7 i = V 1 + ^ *' 



77T 



•i 



\ = \ 



0 



otherwise 



' s(X. )Q. log i 

if Q, >"T7T i - V 1 o + ^ 



7^ 



*i "77^ 



1 



0 



otherwise 



Then y i > i 



E 



,« 1 g(x 1 + 1)1*1 -7175. 






log i 

7I7T 



E 



sC^) ]_ 

g(x. + i)l Q i - ■JT? 



Q i^375 



< M* i?£U since from condition ii) 






E 



g(x 1 ) 1 



g(x i + 1)_ 



M* 



Also 



E 



. V (x 7? ij l^i -> P K - M Tvr 



Thus 



Efl il Q i- a i lP[Q i- a i ] fM * + p] 
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Using the definitions of &. and e. we have 

11 



= 4k exp 



4 Q i log i 2 Q 2 ( i - k + l) log 2 i' 



~T7T 



kVi 



whenever 



Thus 



E 



oa ip 






< 4k exp 



4 log i 2(i - k + l)log i 

~TJK ki 



noting < 1. 



Collecting terms we see that condition b) is satisfied and 



£) < (k + ” i G - k + (M* + fj) _ z 4 



1=1 1 
o 



2 . 



n 



2(i-k+l) log i 4 log i 
ki .1/4 



+ 4Bk Y, e 
i=i 



, 2B 2 m log k n 

171 

n ' 



for all Ge ft . 

We have now given an upper bound for ^(cp, J?) for a H n > an ^- 
remains to find the rate at which this bound goes to zero. Examining 
the various components of the upper bound we have 



(k - 1)IT = 0 | 1 
n l n 
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and : 



2B 

n 





Hence 



2B 

n 



(M* + P) 



n 




log i 

771 



0 



log n| 

■775) 



8B 2 k 

n 




2( i-k+l)., 2. , 4 log i 

ki 108 1 + -^7f 




2B 2 M log^ n _ q| log'" n ] 



n 



W 



n 



k 

1 W 



0) = 0 



~' °i/4- , n ) uniform iy -*- n —• 



n 



Q.E.D. 



We observe that the sum of r independent identically distributed 

random variables, each with density function p 0 (x) = ® X b(®)g( x )^ bas 

the density function f^(y) = 0 y h r (0 )g^ ~ ^ (y) , where g^ is the 

r-fold convolution of g. Thus the density function of the sum has 

the same form, and if conditions ii), iii), and iv) are satisfied for 
(r) 

f> '(y) then theorem 3) may be used even if the original problem is 

modified to allow r independent observations for each 0^, observing 

that the sum is sufficient for 0^. For the geometric, negative 

(r) 

binomial, and Poisson families g satisfies the conditions. 
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B. 



The modified negative binomial distribution. 



Let P[X = x|0] = 



P e (x) = 



a + x - 1 



+ e 



e 



a + 0 



x = 0, 1, 2, . . ♦ a > 0 0 < 0 < £ < «> . 



This reparameterization of the negative binomial is of interest for two 
reasons. First V a,E[x|0] = G unlike the usual parameterization. 
Secondly the form of the decision procedure is different than that 
usually encountered. 

In this and the following sections we shall not give as many 
details as in Section A. The method of proving asymptotic optimality is 
similar in each of these sections, and may be summarized as follows: 



k ^(2^) 

Since \|r.(x, ) = — ■> r when Q.(x ) >0, we seek an estimator 

i ~k 1 K 

k P i ( ^k ) 

cp^x^) which is for "most" x^ e l ua l to a ratio j , such that 

^[1-^)1^ = Xj^] = Q|_ k (25 k ) and such that E[P 1 _ k (^)|x^ = x^.] 

= k_(x . Then using the methods of Section A the functions i ^ 

and ^ may be obtained for condition a) of theorem 2). It is then 

only necessary to show either condition b) or b 1 ) holds. 



kl 



Let: 



1 if x^ = 



- 






g(i, J) = 



0 otherwise 



ja + i - 1 
i 



i a+ jij -1 ) 



i = 0, 1, .. 

J = 1, 2, . 



g(x^., t) if there exists t = 1, 2, .. 

■ ( x i» ••• > x k-i> \ + ^ 



z j ( 2k ) 



such that 



= < 



otherwise 






l 

J=k J 
i - k + 1 



^ " 



£ 

.1=* J 

i - k + 1 



p i(ik> 



if P 1 (x k ) > 0 



p fe> = I 



0 



otherwise 



k2 



p 

2 



if i = 1, . . . , k - 1 



^i(— i.) = { P f ( ^i) if 0 < Pf(xJ) <P i = k, k + 1, ... 



P if P < P*(X k ) 



i = k, k + 1, ... . 



It k It 

Using the above definitions we shall show that (£ = (cp , cp^, ... ) is 
uniformly asymptotically optimal of k th order and 0) < B(k, n) 



|log k n 



i k 



Recall «,(**> - ~.'Y~ 1 JI P e y t ) 

J=k £=1 j-k+i 



Q i ( ^k } = T~ - F + i 'I p « (x " } 



+ 1 J=k J i=l Vk+i 



We observe there exists a set R. of k dimensional vectors such that 

i 

PfX^eR.] = 1 and x^eR^ => Q^(x^) > 0. Thus for i > k and x^eR , 
k x Q i ( ^k ) 

i i (x k ) = y . For i > 2k, Xk eR i> J = k, . . . , i - k we have 



E[Y (Xj)!^ = xj = ]] P© ( X P 

j 1 1 ^ /=i Vk + * 



b3 



We also have 



k-1 



E [z j( ^)|x^ . , t ] = £ g( v t)p (Xk + t) jt p 9 ^ 



(x#) 



J-k+i 



But 



Z g( x > t)p 0 (x + t) 
t=l 



00 

Z a 

t=l 



a + x - 1 



* ap e (x) £ 
t=l 

“ 0 Pfl( x ) . 



6 



a + 0 
it 



6 



a + 0 



x+t 



a + Q\ 



Hence : 



E t P iO£)|j£ - ly) 



r^rr + \ * i , .1 

j=i-k+l 



E fP ± (^)l^ = 



Z 2k + 1 A* (y ) + 

- k + 1 ^i-k'^k^ i 






The remainder of the proof follows that in Section A) 
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C. The binomial distribution. 

Let P[X = x|0] = p„(x) = ( a )0 X (l - 0) a X x = 0, 1, . . . , a where 
a is a known positive integer and 0 < 6 < 1* For this family it is 
necessary to modify slightly the definition of asymptotic optimality<> 
Rohbins [ 5 ] and others have demonstrated why this modification is neces- 
sary* Let R. (G ) be the k^ standard as defined in definition 3) 
K,a — n 

k 

with the parameter value a* We shall develop a procedure (£ such that 

c) lim (sup [R n (jp k , 0) - ^(0^]) < 0 . 

n-» 00 6 ’ 

Such a procedure will be said to have property c). In addition we shall 
show R n (9 k , 0) - a 1 ( j f n ) < B(k, n) = o| ^ Q ^y^ | uniformly in 0* 

We shall first exhibit a procedure having property c)* We shall 
then introduce a new procedure which not only has property c) but for 
most £ actually improves upon the original procedure at each stage and 
produces strict inequality in equation c) * 

We first assume that corresponding to every observation X we have 
available the related observation X’ which would have resulted had we 
observed a binomial random variable with parameter a - 1* For example, 
if X is the number of successes in a independent Bernoulli trials 
with probability G of success, then X’ is the number of successes in 
the first a - 1 of these a trials* While in most situations this 
assumption will hold, we shall see later that it will not be needed* 
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hd > 



Let : 



-i,v 



Y M> 



(x i-k + l> 


• ° • y 


x i> 


< x i-kfl’ 


# * « ^ 


X! . 
1-. 


/ 

1 if 


xk 

-j,o 


= ^k 


I 0 elsewhere 


\ + 1 


if 


X* . 

J J 1 



if 



z .^ - 



otherwise 



I 

- fhrr - 



| Zj(^> 

= a?i - k + l) 



P i^> 

w 



p fe> - { 



if FJ.Xj,) >0 



0 otherwise 



v = 0 
v = 1 

> Vi’ \ + 11 



k6 






1 

2 



if i = 1, ... , k - 1 








1 



if 0 < P*(X^ Q ) <1 i = k, k + l, . . . 
if 1 < P*rf J i = k, k + 1. ... 

i i,0' 7 7 



We shall now show - ( cp^ ; cp^ ... ) has property c) . We shall 

proceed as in the previous examples, noting that theorem 2) is still true 
when the property of asymptotic optimality is replaced by property c). 



V i > 2k let : 



Q i^-^ ' i - k + 1 X 



k 

TT 


a - 1 


L l i 


x * 



q ^ ( i - Q \a-l x. 

j-k+Z 1 'j-k+i' 



R. be a set of k dimensional vectors such that 

l 

q€R.] = 1 and ^k eR i ^ Q i^^ >0 ‘ 

For x^cR^ ve have for the parameter a - 1: 



+*(*,) - 



i - k + 1 



i k 

i e, n 

J=k 0 /=: 



t [ a - 1 
iill x 



Q. 



"i / 



"jW 1 



a-l-x 






i 
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Now for j k, ... , i - k 



k 



, / 



*=11 



a - 11 x. 



a-1-; 



e [ v (^) i ^ = n >^,<1 1 



x.* I j-k+i' j-k+,0' 



E[Z (X^)|xJ --- 3^1 



\ x. +1 a-x, -1 k-1 a - l' x 

)e . (l - e.) * 



- <\ + 



\ * 1 < 



I J 






X A 



V-W 1 



0 .] 

j-k+/ ; 



k 



= aQ 



a - 1\ x. a-l-x« 

^ j-k+i ^ 1 " 6 j-k+Jl) 

X £ 



Thus arguing as in Section A) 



.life) - (h + 



-f(i-k+l)£\+lj-5. 
k a i 



for 5 . > t 



i — i - k + 1 



We now look for an upper bound for the quantity 

We shall show an upper bound is 



n r i k 

I P 



i=k 



i n p e ^i- k+ i ) < i€ 

j=k i=l J-kH J 



(a + 1)V. 



n i k 



& p U & - W w " 1 J 



“- 1 -*8 
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a 

- I • 

x =0 x, =0 i=k 
1 k 



a n r i k 



E E I 



L j=k J=1 ^j-k +i 



P a (xj, i € 



L J a 

=1 i-k+i 



(x i } • 



For any fixed x^* x^ = 0, 1, ... , a i = 1, 2, ... , k. There exists 
a subsequence of integers (i } (possibly finite) such that 



I ft 



l 

j=k i=l '"'j-k+i 



(xj < i € ^ i = i for some v = 1, 2, ... . Let 



0 



i =0. But for all n > k there exists a v such that 
o — n 

i < n =3> v < v and : 

v — - n 



n 

I P 

i=k 



z ft 



I 

j=k i=l ~ j-k+i 



(xj, i 



. € 



ft 



. ,. u p 



£=1 i-k+i 



n k 

< E [1 P 0 ( x i) < \ 5 n& • 

i=k £=1 i-k+i n 



Since there are (a + l) k different t ° sum over, we have shown 

the claimed upper bound holds. 

It is now easy to show, using an argument similar to that in 
Section A, that an upper bound for R (cp, Of) 



k - 1 , 2/ . , \ , 4 ^ log i 

n , f ~lfi 



+ 



n n o 



i=i n 
o 



n 



, 8k r 
+ — I ex P 
i=i 

o 



X log i 2(1 - k + l) 2 



u i 



T7T 



ki 



log i 



2(a + 1) 

— I7T- 

n 



k9 



The 



and clearly this upper bound is uniformly of the order * 

n ' 

desired conclusion follows. 

In the above procedure we chose at times to neglect the results of 
th 

the "a trial” in many of the observations. This choice of which 
information to neglect was quite arbitrary, and it is easily seen that 
the above proof does not depend on which trial's information was neglected. 
We may thus conclude that if in some situation the related observation 
X 1 is not obtainable, we may construct a new X 1 which will do as well. 
An example will illustrate. Suppose a = 17 and X = 10. With the aid 
of some random device we let 



X 1 = < 



9 with probability 10/17 
10 with probability 7/l7 



This X 1 will work as well as the original X'. 

^ k 

We shall now exhibit a procedure Cj) which improves upon cp , At 
the i th stage of the decision problem we could have defined qh in 
any one of several different ways, depending on what information we 



chose to neglect. For fixed i > k and x^ there are a ways 



to 



define Q and hence Thus there are a 



k( i-k+l) 



ways to 
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define 



and 



P i (x k ) . Similarly there are a^ P ways to define Z^(x^) 

hence a^ ^ ways to define P^x^.) . Thus there are 

a (2k l)(i k+l) wa y S d e fi ne p*( 3 c^) . Prom the above, and observing 

the X^ _ used in P*(X^ .) could have different definitions, we 

see there are at least a^^ ^ k+l)+k wa y S define the random 

variable cp^(X^ ) . Most of these different definitions will result in 

essentially the same estimator for large i. We may obtain an improved 

procedure, however, by considering some of them. 

k 



We define \ u = 1, 

-l, (u) 



as follows: Let u = 1, ... , a 



be an indexing of the a distinct k-tuples each of whose elements are 
integers from the set (l, 2, ... , a). Let the u^ k- tuple be 



(t ... , t . ) . Let X^ l = 1, 
N u,l' 7 u,k' 7 



y a be the random variable 
th 



derived from X by not counting the result of the Z trial. For 



example, X 



(a) 



equals the previously defined X 1 . We now define 



Jk K i) K 2^ K k^ 

XV^ (u) to be the random vector (X._^ , X._^ , • • • , X. u ' ). 



1 

2 



if 



Let : 





P*(X^ , x 



) 



if 



1 if 



i — 1 , • . • y k " 1 

0 < P *(2^( u )) < 1 i - k, k + 1, . 

1 < P i ( ^(u) } i = k, k + 1, . 
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Let: R(q>, 0.) = E[(cp(X.) - 0.) ] 



q?(X. ) = V cp k (x.) 
T i — i k — l.u—i 

a u=.L 



$ = ( 9 -^ <P 2 , *** ) 



We shall now show V £> V i RCt^, 0^) < R^^, 0^) where cp^ is 
as previously defined « We first observe that the cp^ ^(X^) are 
identically distributed as cp^(X^). Then, supressing the superscript k, 
we have 



R(?i, £ ± ) = e[(5.(x.) - 0 .) 2 ] 

= E[? 2 ] - 20 i E[? 1 ] + e 2 



2 k 



a 



k 



I Etc J + 2 I E [ 9 9, 1 



L n 1 > u 

L U=1 7 



u <v 



1,U 1,V - 



26 . a 



r Z E[(p. ] + 0 2 

k ft Lr i,u J i 



a u=l 



= Ei(cp i - ©.r] - 



r a k - 1 2 



^ a 



F" ^ - -2k 5 E[ ”i,u *i,yl 
a u< v 7 



= R(cp.. 0.) - -Jr- Z Ercp? ] ~ 2E[cp . <p. ] + E[cp 2 ] 

—i' 2 k “ lT i.u T i,u Y i,v h.v 

a u<v u 7 7 - 



- b(v £,) - -s Z - c f] 

a u<v 7 7 



< *(9^ -±) 



we notice strict inequality holds unless P[cp^ (Xp = 9 ^ v (X^)l - 1 
for all u, v 1 < u , v < a « Thus we have strict inequality holding 
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unless a = 1 or unless 0^ is composed of l's and 0's* To 
investigate the asymptotic properties of ^ ve observe: 



u, {i l r( 5., fl ± ) - R* ^ 

n-* oo ^ ’ 



n 



n 



- lto Xn .1 t R K> 4> - A’ 4)1 .1 E <V - 9 i> ' 4, a- A 

n-> i=l i=l 

< lim -j~ £ [R( 9 ^, j§^) - R( 9 ^, 6/) r 6 ince ^ has property c) 

n-» 0 ° i=l ^ 

{- i .1 -tk 5 

1=1 a u<v ' 7 J 



= lim 
n-> oo 

< 0 



Hence ^ has property c)» In order for strict inequality to hold it is 
sufficient that there exists e > 0 such that, with the possible 
exception of a finite number of values of i, EeUT- - <p. I] > e. 

* ~ 1 lU 1 • v — 

u<v 7 7 

If a = 1 this condition is never satisfied. For a > 1 , however, and 

for a large class of _0 such an € will exist. Let ft* be the set 

of g such that V _0^* there exist such that 

O<|^< 0 ^<^ 2<1 for all but finitely many i and such that the 

first order empirical distribution function of .0 does not tend in 

the limit to the distribution function of a degenerate random variable. 

It may then be shown that a > 1 j?«ft* implies 

£ E[ j 9 ^ ^ - qp i ^| ] > € > 0 for all except possibly a finite number 

u< v ,U >Y 

of i. 
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Since the sum of independent identically distributed binomial 

random variables is again a binomial random variable, and since the sum 

is a sufficient statistic for 0, it is clear the methods of this 

section can be applied to the case of r independent observations for 

each 0 . . 

l 

D* The normal distribution 



Let 



P[X < x| 0] = 



p 0 (t)dt 




-00 



-oo < X < 00 



for o > 0 and ~oo < a < 0 < (3 < oo 



For the present we assume a is known, although later we shall modify 
this assumption somewhat. As we shall see, the estimation procedure 
in the continuous case is similar to that in the discrete case, Without 
loss of generality we take a = 1. We fix k > 1, 

Let { c . , i = 1, 2, o • o ) be a sequence of positive numbers such 



that lim c. log i = 0 and lim i(c.) 

i— > oo i— > oo 



Ij-(k-KL) 
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e. be the k dimensional vector consisting of zeros for 
all components except for the k^ which is equal to one, 



v<^ - 



1 if - c i < ^ t-l, • • . k 

0 otherwise 






(1 - k + 1K2C.)' 



gi(l k ) = 



f l<i k + c i— k ) - f i<& ' C A> 



2c. 

l 



g,(y k ) 

+ 7 TZ A if ffjfc) > 0 and 1 = k ; k + •' 



w 



' { 



y k 



otherwise 



a 



if p*(xb <a 

1— l 7 — 



qp^(x ± ) ={ Pf(xJ) if a < P£(x£) <p 
P if P < Pf(X^) 



k k k 

We shall now prove the decision procedure qp = (cp^, cp , 



) is 



uniformly asymptotically optimal of order 
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We shall show the conditions of the corollary to theorem. 2) 



are 



satisfied. 



f 'I jEfri-Vk 



Let. - I . Is) 

J-K 



6Q, (j k ) 

W - -5T 



± 2 ~2 



Qj(j k ) = I 0 j |s| e 



j=k 



Now for any 



k QHy k ) 

■ w 



But Q^(j k ) = 






2 "2 „^ y /' e .j-k-hP 



i-1 



“ ' y A^k> + Q i 



„ k, , , «&> 

Hence * y k + Q~^y 
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Let: m i(j k ) = E[f.(X^)|xJ = Ik ] 



Then 



m 



Now: 



z ih k ) 



i-k 

I 

j=k 



_i y (y* _e ) 2 .1 y ( y _e \ 2 

2 ^<3 j-k+r 2 j-k+i ; ■ 



- e 



(i - k + l)(2ir) k//2 



for some y* such that y. - c. < y* . < y„ + c. 

- k, j * l & , j * l 

= 1, ... , k. What particular y* . is intended 

&■) J 

will he clear from the context. 



y £ +C T 1 . 

i-k k p -y( t - e i . k +«) 

L<lk> “ TT^TTTy J Jj2 T J ~j= e 

y r c i 



dt 



Q i-k^k^ , x 

i-k+1 z i -^k 



where the y* in z. are those vectors whose components 

~k, j l 

arise from use of the mean value theorem* 



k oo 

for V i = 2k, 2k + 1, ... , y oR > $€ ^ 
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f .(X 1 ?) - T — U ~-r 1 > 5.|x^ = 
i —i i - k + 1 — l 1 — i 



^k 



< p 



i - k + 1 
i - 2k + 1 



- “iM - T 



k + 1 
2k + 1 





k 

i - k + 1 



but from lemma 3) this probability is 



< 2k exp 2 (l ~ l ~ ^ (5. - | z. | - — | V ' 1 ) 2 (2c.) 2k 



^ f^2(k+l) 2k c ^2k+l/ i - k + l s 2k /c , u 2 

< 2k exp ^2 'c. 5 i - 2 ( ) Ci (5. - j z i j ) 



provided 5 . - | z . | - 



i 1 i 1 i-k+1 — 



> 0 



In a similar manner: 

Let : 

z i ( ik + “A 5 ■ z i ( ^k 



;<yj = 



c .e. ) 
l— k 



i — k 



= 7 





2c. 




l 


1 1 


f Q i-k ( 2k + c i-k> - 9 i-k ( *k ' 'A 1 


i - k + 1| 


1 



9 i-k ( ik ) 



Then 



k , k m i ( 2k + c i-k^ " m i ( ^k " c i-k^ 

Etg^x*) |x^ = jr k l = i — 



9 i-k<* k) 



= i I k + i + + “i^k) • 
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Hence : 



,.k, . ,„k 

g i ( 2i) - i - k + i l ^ e i'2i ‘ ik 



< ak exp -i - 2(1 ~ k k * ^ (e t - |zj| - Ujl ■ i . it + i 



k 2 < 2 c i> 

) — 5 “ 



2(k+l) 



„ J/^ \2(k+l) „2k+l 2(k+l) i - k + 1/ |,| i i >,21, 

< 2k exp l(2 Cl ) e i ~ 2 c i k ' e i " l z il " I I ) J • 



provided e 1 - | z^| - IqJ - - ^ — >0 



Thus, using an argument similar to that used in proving lemma 4 ) and 
letting B = max[|o:|, |(3|], we have: 



K<2i> - *lM > X ' ^> ~ (€l + ^ + ^k 15 !^-! = \ 



i - k + 1 



f^2(k+l) 2k c ^2k+l/ i - k + 1^ 2k /c 1 _ 1 v 
< 2k exp -j 2 / c i 5 1 - 2 ( g Jc.. (B ± - | z ± | ) 



x 2(k+l) ^2k+l 2(k+l) i - k + 1 

+ 2k exp l(2c.) v e i " 2 c i k 



(e ± " l z ll - kj.1)' 



provided 8 ± - | z ± | - ± _ k + ± ^ 



> 0 



e . - z 1 
1 1 1 1 



k 



^i' i - k + 1 — 



> 0 
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To complete the proof it is sufficient to show that for some functions 

®j) and 5 i(^ k ^ £ ± )> such that either e ± > |z|| + |q ± | + |~~ i 

i i k 

or = 0 and either 8^ > |z^| + — ^ + ^ or 5^ = 0, the following 

limits hold uniformly in 6_. 



i) lira E 

i-> oo 



it # 2 -r ■■ • 



ii) lim E 
°° 



,Y ( x" (B * |x ii> 8 i(^]=° 



iii) lim E 



i— » oo L 



-ic^ k (8 i (X^)-|z i (xJ)ly 






= 0 



iv) lim E 



-ie^ k+1 ^(£ 1 (5C^ ) - 1 | - 1 %<>$) I Y 



i-> “ L 



€, “ Z 



l z il - li.l > 



11 - (i '-"k + i)^J 



v) lim P 

i— > eo 



8 a (xJ) - |z ± (xj)l 



(i - k + 1) 



+ l) 1 ^- 



» 0 



Vi ) - 1*!^ 1 - "^ 7777 1 ) 



175. 



= o 



6 o 



Let 



(i - k + l)log i 



if 



k 



I 




1 



0 



otherwise 





if 



(i - k + l) log i - + l q il + (j._ k+ j.) 1 / 1 *- 



(i - k + l) log i 



0 



otherwise 



Since there exists U < <» such that E[|X|] < U for all Qett, the first 
two limits hold uniformly. The third and fourth also hold, recalling 
lim ic^ 15 * 1 ) = 

i-> oo 1 

Since V > 0 there exists s(n) < <» such that P[|x| > S] < t) for 
all , we have: 




< 



k 



175 < s * - 



k + 1 - (1 - i]) k = 



(i - k + l) 



6 1 






1 k ' 
i . ± ik/2 “2^ (X i -k+i 



J=k 



2tc J 



i - k + 1 



< ^ . l o g i | | logl 

(i - k + l) 



^ X i-k+i^ — ® >k 



+ i - (i - n) 



< P 



x tk/2 -|k(S+B)‘ 



2jtj 



< (TT!7) i 75 + Ul|log 1 K-*“ I ss . 



+ 1 - (1 " V) 



Clearly the fifth limit will hold if we show lim | I lo £ i = 0 

i-» oo 

uniformly in ©eft and |y^|<S i = 1, ... , k. By a similar argument, 
to prove the sixth limit holds we need only to show 

lim (l z i(j k ) + I ^i^k) I ) log 1 = 0 uniformly for ©e ft and |yj < S 

i-» oo 

£ ~ 1, • • « , k« 

We first consider lim | z i (^j c ) I log i* For Yp ~ c i < Y^ < Yp + 



i— > oo 



i J £ J £ i 



^ ■ 1, • • • y k; y j k, • • • y i 



3 - e 



» e 



Z (y|- 0 j- k+ i ) 2 t Z [ (yr e j-k+P 2 " (y l“ 0 j-k+i )2:i 

i=1 | |i - e i=1 | 
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But 



l (y i “ Vk+P 2 ■ (y t ' e j-k+i )2 l ^ l y i “ ylHy.0 + y| - 20 j_ k -J 



< c i (2S + 2B + Cl ) for all 0 j_ k+i efl 



00 V 



Now 



|x| < 1=» |l - e | = | £ ~ I < |x| (e - l) 

V =1 V i 



Thus 



'i^k^ I 



i-kr “ 2 i | 1 ^ y l,j" 0 j-k+i^ 



1-kr 



- e 






(i - k + 1)(2*) 



k/2 - 



M S + B + c.)(e - l) k for all £ and i suffi- 

( 2 *) k/2 1 1 

clently large. 

But since lira c^ log i =? 0 we have shown lim Iz^Jlog i = 0 for all 



i— » oo 



i-» oo 



£ and |y.| < S £ «= 1 , ... , k. 

We now consider lim | z ' | log i. This limit is more difficult to 

i-» oo 1 

evaluate. By examining the argument leading to the consideration of this 

limit it is clear that we need only show lim | z_jj log i = 0 for such that 

i-> oo 

lyj < S for £ = 1 , ... , k and y^ 5^ 6^ for j = k, k + 1, ... . We 
shall use lemma 5 ). It is clear that there exists M < oo such that 
°M = oo ) for all 0 e fi . Thus we have for i > k and y^ such 

that |yg| < S and y^ / £ = 1 , ... , k j = k, k + 1, ... 
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I *i<2 k )| " 



i-k '^?i (3l '^- k+j)2 ( -l<»r-v 

e le 

J=k 






2c i (i - k + l)(2rt) 



k/2 



1 k ~ 1 ? 

• v - 2»^ ^i~^j-k+i^ / (y +c -0 ) 2 --{y -c -0 V 

i-k ^i=i J K * 2'^k i j' J^k i V 

V e 1 e ° - e u 

j=k jT~ T^T oI^T^T^ - 



2c i (i - k + l)(2rt) 



where y £ - c ± < yj < y f + c ± y** = y* + 2p ijj c i 



*£** = y k " 2p 2,j C i 



_ |(B+S) 2 

li _ 4v2rt M e 



•v,J 



2 < < 



Tfy 



k 8 j 



v = 1, 



and 



Kl - £ 



1 ^ 2 k-1 

l-k "2j? 1 (yre J-w ) -rX (y J- jr i )( jJ*y 1 -29j. tl .p 

e -0—1 



k^ 



J=k (i - k + l)(2 I t) 1V£: 2c i 



-| (y k*- y k )(y k ,+y k - 2e j ) -§ <y r*- y k )(y ir* , 'V 2e ,i ) 

e - e 



-i(c 2 +2c .y, -2c .0 .) 2c.(y.-0.) 

e ^ 1 1 k 1 J ( 1 - e 1 k j ) 
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We shall now consider various parts of the right hand side of the above 
inequality. 



V j = k, ... , i and i sufficiently large: 

- \ i. + y < 



a) Let £ = £ 

1,3 V =1 



-|V 



i=l 



2 Vk+.P 



vl 



then 1^1 < c ± ( e - l)(S + B + c ± )(k - l) 



,k-l 






and e 



-i*l 



= 1 + t 



1,3 



b) Let t 



2,J 



l 

V=1 



/ 1 2 
( “ 2 C i 



c.y. + c .9 ) 
i k i y 



vl 



then \i \ < c i (e - l)(S + B + c ± ) 



^^Vt' 2o iV . 1 



and e 



+ i 



2,J 



c) Let £ 



3,J 



» c 1 t g (y k - «,)] 

V=1 



V+l 



(v + l)i 



then | j| < c i (e 



2(B+S) 



1) 



and 



2cj(y k -e ) 

* ' 2 < y * - V - • 
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00 



tf) Let 1 = £ 

4 ' J V=2 



[- ^(y,,** - yj(y£* + 



2 k k k •'k 



y^ - 



2a)] 



c. vi 

i 



oo [_ -|(y*-** _ y k )(y k *** + y k 






v=2 v : 



( S+B+ c . ) 

then | | < 2c i (e - l) 

-§(y k **-y k )(y** + y^ 20 *) "i^ y k*^“ y k) ( y^^+y k -2e , ) 

e J - e J 

and 

c . 

i 

-ft -ft 

- (y k - y k )(yk* + y k - 20 j) + (y$^- y k )(y^** + y k - 20 j) 

• 2T t ~~ ~ + ? 4,J • 



e) Let 5 5>j - 2= 1 (p^ J - p^j) 



th6n I £ 5 ^ I — 2c : 



and 



(y£* ~ y k )(y k* 4 y k - 2 V - ' y k )(y T* + ^ 



20 j } 



2c. 

1 



(y**) 2 - (y***) 2 - 20j(y^ - y k ***) 
— 



< y k + 2P 1,.1 C 1 )2 - (y k - 2P 2„1 C 1 )2 ' '’ViW.J * 



2c. 

l 



4c i y k< p i„i + Pg„i> + 4c i(^,.i ' + Pa.p 



2c. 

i 



“ 2(y k - V ( %j + + h,i 
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1 



f> Let i 6>} . p 1;J + p 2)j - 



then U 6 J< |p 1;J -|l + Ip 2 ,j -\\ < 3 | y^ 2 - Sj 



8/5 I* 1 * 3 ) 2 

e 



and 2(y k - SjKp^j + P 2;J ) - 2(y k - 8j)(l 5 6>J ) 



" 2(y k ' V + ? 7,j 



~~ I v,[ < a# 



Using the above six results we have for sufficiently large i: 



2(2n) k//2 ( i - k + l) 

< M*c^ where M* is some finite constant independent of £, i, 
and jr k . 



Hence lim | zjj log i = 0 uniformly in 0 and such that |yj < S 



1— » 00 



& = 1, . .. , k and y ^ 0, j = k, k + 1, ... 

J 



We now consider lim |q i (y k )|log i for | J < S i = 1, k 



i-> 00 
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,k-l 



mi :: 

2(1 - k + l)c i 



i - 



l-k k/2 '2,3 y i" S j-k+^ ) 

TTi l k < y k - V Isl * 



So 



. i-k -iVv 0 / _ -l<V c i- 8 j ): 
I I - i - k + 1 £ 

J=k 



dc . 



-i<y -0 ) 2 

0 \Jl. W if 



+ ( y k- e j)« 2 k J ' 



l ( ‘ : i +2c i y k' 2c i £l j ) •| (c i' 2o i y k^ c i e j) 



1, 2 



< — 



i-k 

bn I 

k 1 J=k 



- e 



2c. 

i 



+ < y k - v 



Let £ . = 



I 

V=2 



t~ c i<T * y k - e p ] _ “ - y k + V 1 



2c ± vl 



I 

V=2 



2c. v! 

l 



then 



B+S+c. 

UJ 5 2c i( e 1 - x ) 
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-^(c 2 +2c.y. -2c.0 . -i(c7~2c.y. +2c .0 .) 

2 1 l^k i j 2' i :rk l j 



1/ 2 



and 



- e 
2c. 

l 



+ e j + 



B+S+c . 

Hence 5 2c^(e 1 - l) for all £, ^ such that |y^| < S; 

and lim |q i (^)|log i = 0 uniformly in 0 as desired. 

i-» oo 

This completes the proof that the decision procedure cp is 

t h k. 

uniformly asymptotically optimal of k order. was defined for the 

case a = 1. If we had kept arbitrary a then (£ would have been 
defined in the same manner except that would have been defined 

° 2[f i^k + c i-k) ■ h^k ■ c i-k^ ] 

as — — - — — — * If we relax the assumption a 

1 

known to the assumption a unknown but equal for all observations, then 

it may be shown that if o\ is an estimate which converges in probability 

2 2 /\2 
to o uniformly in £, we may replace a with in the definition 

of an( 3- th e resulting decision procedure is still uniformly 

asymptotically optimalo 

If the problem is modified to allow r independent observations 
for each 6,, then since the sum of these r observations is sufficient 

i 7 

for 6L and also normally distributed, the above procedure will still 

apply. We note in this case that if the common variance is unknown, 

^2 

then for each i the usual estimate a 7 is independent of 0^ and 

^2 2 
a. is a consistent estimate for a . 
i 



- I 

n i= i 



69 



E* The gamma distribution* 



Let P[X<x|6] = / p (t)dt 



X 



9 (8t) a - 1 e" 0t at 



7TS7 



0 < X < 00 



for a > 0, O<a<0<p< 



We assume a is known, and fix k > 1. 



Let {c.), e , Y , , f., and g. be defined as in Section D. 
l — K J , I 1 1 

Let: 



a - 1 



^ k ) - 



a 
yv 



g l ( ^ , 

f i ( 2 k > 



if f.(y ) > 0 and 1 = k, k + 1, 

i ■ K 



otherwise 



<< 2 i> 



Jtv 



y. if P*(X7) < a 

{ pf(^) if a < p*(x^) < (3 

Ip if P < P*f-(x^) 



— 1—1 



k k k 

We shall prove the decision procedure (£ = (cp^, cp^, 



th 



uniformly asymptotically optimal of k " order 



) is 
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We shall show the conditions of the corollary to theorem 2) are 
satisfied. Since much of the argument is similar to that in the normal 
case we shall omit many of the intermediate steps. 



Let: Q ftZ k ) = 



i 

£ 

j=k 



ft e j-k+i a-1 " y i e j-k +i 






- 



er . -y,e 



J 



V' Q ft "j-k+i .a-1 _ J r j-k+i 



Then for all such that y 0 > 0 i = 1, . . . , k , 



i 



k SiteJ 

♦A> ■ . 



But 



Q i ( ^> * I 

j=k 



k-i e 



J-k+i a-1 “ y /j-k+i 
iL —~ Ye e 



LJtii na) 'i 



e : -yve 



TT^T e 



k j 



(a 



i-P 



i)yr" - e ,y 



a-1 

j J k 



a-1 



Qi(z k ) - Qt(j k )< 
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Hence t^( Z k ) = — 



^1 Q lfa) 

y k “ Qi(z k ) 



Lett m ite k ) = E f f i(2^)l2^ ■ Z k l 



i-k 



i - k + 1 £ 

j=k 



k e 



=1 



■fer 'n,/' 1 



ft x %r <^> a_1 •' yj9j - kfi 



for j[* j such that y { - ^ < y < Y t + <=* 



Then: 



Q, v (zJ 

i^k> = r-YV-i + z i ( V for clUtable n,y 



oc 

and for i = k, k + 1 , ... y^. such that y^ > 0 i = 1 , ... , k ©eft 



I [ |f i ( ^ ) " TT-tTl I - s i ' 4 * 2 k 



^ 01 Jn ^ 2 (k+ 1 ) 2 k^ ^ 2 k+± i - k + 1 2 k, c 

- 1 li k l l 



provided S ± - | z ± | - ~j~__ > 0 

r 



vhere B - max 



V ( - U " 



Z il } 
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Lezt z^(j k ) = 



z . ( y, + c . e, ) - z . ( y. - c . e, ) 
1 £ k l— k' 1 % i— k' 



2c. 



, , . Q i-k ( 2 k * 'A 1 • «i-k ( 2 k ■ °i-k^ , 

«i l V * rnrr: ^ — - — «U ( 2k> 



Then: 



* I k ! > rr^i + 9iU k ) + *i<* k ) 



and f cr i = k ; k + 1 $ * . . y^ such that y^ > 0 £ = 1, * • . , k 9e fl 



k k 

i S i ( 2i> - 1 - k t l 1 ^ e i^i = 2k 



< 2k exp jBxtgc.) 2 ' 11 * 1 ^. - 2 2k+1 c 21 k '‘' 1) 1 • k «■ L . | 2 .| . IqJ) 2 } 



provided e ± - |z‘| - J q. ± | - y-^ 



kB* 



k + 1 — 0 



k _x dp e (t) 

where B* < °° is such that max (Pg(x)) — 

x ; y> 0 

Thus we have : 



< B* 



t.=y 



P ^|<p.(X.) - \|r . ( X . ) | > ( e - + PB. - l a '■ X U .)jx k = y. 

L 1 1-1 1 -1 1 - Q i (y k ) 11 y,. 1 '-1 ^k 



^ ™ ; ^2(k+l) 2k c „2k+l i - k - 1 2k /c , r ,2 

< 2k exp -{B2 'c. b ± - 2 — - c i (& i - | z ± | ) 



B*(2c.) 2(k+l) e. - 2 2k+1 c 2(k+l) - 1 ~ * + te. 



+ 2k exp -jB*(2c 1 )" v “ ""c. - 2^“ - "■ k "( € ± - jz'| - | q ± | ) 



,2 
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provided 5 ± - | z ± | - j 7 -^ - 5 ; > 0 



K\ “ <JU 



kE* 



i - k + 1 - 



> 0 



To complete the proof it is sufficient to show that, for appropriate 
6^ and e the limits i), iii), iv), v) and vi) listed in Section D 
hold uniformly in Q_, and that 



ii») lim E 

i-» <*> 



QjtxJ) X i 11 



= 0 uniformly in 9 . 



Let j 



y *W ,, y k Q i^k> . , , . 

(i - k + l)log i " (i - k + 1) log i — ' i' 



kB 



(i - k + 1) 



175 



8 i ( *k> “ \ 



0 



otherwise 



,, Q i . , , . 

( i - k + l)log i ( i - k + l)log i — ' z i ' 1 "i 



k E-- 

(i - k + l) 



1 A 



‘i^k 5 - 



I 



0 



otherwise 
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Then limits i), iii), and iv) clearly hold. Since E[X] <— for 
all dett limit ii') also holds. V 1 > 0 there exist s(t|), S(h) 

such that 0 < s < S < » } p[x < s] < | and P[X > S] < | for all 
. Hence it remains only to show: 



lim | 


l z i(j k )l 


log i = 


0 


uniformly in 


9 


and 


2 k such 


thal 


i— > oo 
























S < y £ < S A 


~ 


1^ • «* ' 


. , k 




lim | 
i — > °° 


Z iM 


log i = 


0 


uniformly in 


e 


and 


such 


tha*. 










s < y^ < S and 


y k * 


a-1 












i = i, ... , k 


II 

*•“3 


, k + 1, . 


» • » 


lim | 


q-i(z k )l 


log i = 


0 


uniformly in 


9 


and 


^ such 


tha , 


i-» “ 
























s < y £ < S l 


= 


1; • • 


. , k . 





Now 



i-k 



W I s - ,-k — I 



t 1 ! •)-« 



j=k (f(a)) 



k 



i (y? - 



^a-l e j-k+i 



^ a-1 y <? t j-k+i 

y i e 
=1 1 
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v j = k, ... 



i - k 



y| 

^ v = i -^ 



than y t - a. < y* < 7j + Cj «#■ 1 - -± < ^ < 1 + 



c . 
1 

s 



** < T 



so that 



A 



a-1 



yJ 



= A 11 ■ ?i ’ <l 



a-l 



- i + £- 



=i 



where, for 1 so large that < s. 



|U < 1 + 2 k(a - l) — 

1 — s 



Let £ 0 = 



E 

V~1 



- yp e j > k+i 



VI 



then |^ 2 | < c 1 (e 1 ^ - l) for | c i | < 1 



k 



and e 






- l + £, 



16 



Thus 






/ „\a-l J-k+i 

(y|) e 



- 1 <* 



I 



fcfil 



■k+i 


1 


ft 


n 


L l i 


i y il 



a-1 






ii 



< s^id - yd + y - 1| 
<s a_ 1 (is 1 | + u 2 | + iy 2 |) 



so that 11m |z.(j )|log 1 = 0 uniformly. 
i-» oo 

We now consider 11m |z^(^ k )|log 1. As in the normal case we shall 
i-> ~ 

use lemma 5). 



We have i 



,z i(z k )l 5 (i _ k + i) I 



1-1 j-k+i 



J=k (r(a)) 



rk-1 -v* <9 

Ti (y '?,j )a ’ 1 e ,J J 



-y„ A 



, , - . -v-X-JHfO ■ 

(ypj 8 "" 1 e k ' J j - (y***)®” 1 e <k,J J 



k, j ‘ 



k,j‘ 



2c. 

i 



a-1 " y /j-k+/ 

/i 6 



(y k ♦ y- .- ( ^ )e -i - (y k - y»- V (y *- 0 ^1 



2c. 
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Q k(a-l) A ka i-1 

< E r I 

(i - k + l)(r(a)r J=k 



fw- |Zkj a ' e - (y i,j- y i )e j-k«! 



ZLi 



a-l 



v #*-*, a ’' 1 -( y ***- y *-* )e 

^71 1 e j 



2c , 



- € 



Vv^j 



3,*“1 EL"*1 ^ rs 

I + c 4 1 /y n - C _ j 2c .0 

w k i k i l j 

~n e 



2c, 



where: y*^ - 7* + SP^ ^ " 2 P 2 ,j c i 



4Me 



(p v,J 2 ^ - 3|p' 0 (y k )| - ^ s a “2 



4Mr'(a)e aS c, 



!a - 1 - Vk 1 



V = 1, 2 



We shall now consider various parts of the preceding inequality for some 



fixed J k < j < i - k. To simplify the notation we let y y . 

k > J 

c = c^ 9 = ©j in what follows. We assume sufficiently large t. 



0 Let (1 

- 1 )' 



! _ZLi 

Yo 



then iq^l <| 
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and 



where 



b) Let 



then 



and 



c) Let 



then 



and 



d) Let 



then 






a-1 



- 1 + 5 - 







IU < 



2 2 



a+1 




2p c 

(i +-— ) 



a-1 



2(a - l)p,c 

= i + + i 0 

y 2 




ig < 



2 2 



a+1 



a-1 



1 „v K v I a-1 2p c 

^ = (l — ) .1 

y I y 



2(a - l)p„c 



+ t 3 



- [ a ' ^l- 

' - v ' v My 

' 4 ' Je v! 



bit 1 5 ° 



2 2 



a-1 
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and _ ! ♦ Xa^ila ♦ ^ . 



e) ^ 5 5 ■ v ? 2 r ; ) 



-■£) 
. .. y( 

vl 



then | £ j < c 



2 2 



a-1 



ana | i ^£| a ' 1 . i - - , i)° + { . 

1 y f y 5 



f) Let C » E 
V=2 



00 [gc^ + P 2 ) 0 ]' 



Vl 



then |Cg| < c 2 (e^ - l) 

_(y***_y**)e 2 c( P;l +p 2 )0 



and e 



g) Let £ = 



= e 



= i + 2 C (p 1 + p 2 )e + 



00 [(2p, - l)c0] V 

£ - v . - 

V=1 Vo 



then | t,rj | < c(e^ - l) 

-(y+c-y**)Q ( 2 P 1 _1 ) C0 

and e' y ^^'=e = 1 + £ r 



h) Let Cg = I 



(gee)' 



V=2 



Vl 



8o 



then |^gj < c 2 (e^ - l) 
and e 2c0 « 1 + 2c0 + £g . 



rk-1 



i) Let i g = 



I 

V=1 



£ <n, s - 



vl 



then | £ | < c(e^ k-1 ^ - l) 



and 



jf e -W, 3 - y <>V« . , + 



Thus 




[ILI 

1 y i 



a-1 





e 



_(y***_y**)0- 



-(y+c-y**)0 



y + c 
JL ... 



a-1 



„ I , 



a-1 



2c0~l 



2c 



(1+ £-]_)(!+ t<j) 



2(a-l)p.c 2(a- l)p p c 

i+ + i 2 - (1- - ■ + £ j] (1 + 2c( Pl + P 2 )e + l 6 ) 



2c 



- (1 + t 7 ) 



[i + + ^ - 1 1 - (a ; 1)0 + uu + 2= e + e 8 )] 



2c 



8i 



= J ( p + p — l)(— - 0) + £| where |£| < cM* for some finite 

id y 

const ant M* independent of 
9, or j, 

<|p x -|l 1^— i-e| * |p 2 --|l l^-«l + l!l 

< cM*“* for some finite constant M*' v independent of y^ j9, or 

Hence lim | z^(^) [log i = 0 uniformly in S and jjr for the set of 
i-> oo 

^ of interest o 

It now remains only to show lim |q.(y, )|log i = 0 uniformly for 

, 1 • K. 

i-> 00 



such that s < y^ < S i = 1 ; , k. But 

, i-k l k-1 

I ^(z k ) I 5 i _ k + 1 ( JT 



1 - klk = ie U^ a-l - y /j-k + i|_i 



rT 5T y i 



y# e 



rTaJ 





l y k + c il 


a*' "J- 

e '°i J 


y k - c i 


y e 
y k j 


1 \ 1 


e - 


y k I 



a-l 



c.e. 

„ 1 j 



2c, 



(a - 1) 



y 



- 0 



k 



So it is sufficient to show that s < y < S 0efi 



t a- 1 q a- 1 o 

y + c | „-c0 _ / y - c c0 

y I 6 a-l 



2c 



+ 0 < cK 



where M is some finite constant independent of 9, y, or c. Using 
parts d), e) and slight modifications of h) in the proof of the previous 
limit we have : 
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< cM as desired. This completes the proof that the decision pro- 

k t h 

cedure eg is uniformly asymptotically optimal of k order* 

We note, as in the previous sections, that if the problem is 

modified to allow r independent observations for each 6^ then the 

sum of these observations may be used to obtain an asymptotically 

optimal decision procedure of k^ order. 

We now consider the case in which the other parameter is unknown, 

Q 

that is Pg(x) = ^" e for known X, It may be shown that if 



1 + 



X + 



£i^k> 

f i<2? 



if f . (y. ) > 0 and 
i -k 



i = k, k + 1, . . . 



p I<z k ) 



-< 



1 + Xy otherwise 

K. 

\ 



and all other definitions in this section are unchanged, then the 
resulting Cj)^ is uniformly asymptotically optimal of k^ n order. 
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F ■ 



The non- parametric case. 



We now consider the following problem. Let 3 = {p^( ): 0e ft } 
be a class of probability mass functions, each of which assigns proba- 
bility one to a specified denumerable class % = {x] of real numbers. 

Sc is an arbitrary index set. We assume for each 0 in Sc that P^(°) 
is completely specified. Let h( • ) be a real valued function on 
Let \{e) = E[h(X) | 0] . We assume E[h 2 (x) j 0] < B < «> for all Q in 

G , For some unknown 6 .e Si we observe r + 1 independent identically 

J 

distributed random variables X. ...... X. ... with P[X. = x] 

JA J>r +1 <J;S 

= p. (x) s = 1. ... , r + 1 xe9:. We wish to estimate X(0.) on the 

e j J 

basis of these observations. For example, if h(x) = x then we are 

estimating E[X|0]. If cp. is our estimate we suffer a loss of 

J 

2 

(cp. - X(0.)) . We now assume we are faced with a sequence of such 
J J 

decisions. In other words a sequence ( 1 : j = 1 , 2 , • . . ) is 

J 

00 

selected from ft , For each 0. we have r •+- 1 observations and we 

J 

may use X. to estimate 0., where X, is the j X (r+1) matrix of 
— J J ~ J 

observations (X ). Johns [3] has considered this problem under the 
s , t 

assumption that each 0. is an independent observation of an ^-valued 

J 

random variable © with unknown a priori probability measure G 

defined over a suitable a-algebra of subsets of 1 We shall consider 

the case in which the sequence (0.) is arbitrarily chosen, 

J 

As in the previous cases we need a standard to use in evaluating 
a particular decision procedure. For any 0^e Si n we form the k^ h 
order empirical probability measure G^ such that for any sets 
Sl^, o.o y in the a-algebra, 



8L 



G^fQ , ... , n . ) = — ■ — r [number of i (k < i < n) such that 

nl k n - k + 1 - - 

e. , , yjt n £ = 1, . . ♦ . k] . If we assume for i < k < m that 

{0^; i = 1, . • . ,, m) is a sequence of random variables taking values 

k k 

in fi with © m Having any k-dimensional probability measure G , and 



© is independent of © , 



m 



i* ••• 



© . , then the Bayes estimate for 

m~k' 



M© ) is E[X(© ) |X ] and the Bayes risk is 
m m =m 

R ._(G k ) = E{[X(0 ) - E[M'© )jx k ]] 2 ), where X k is the k X (r*l) 
r+1 m m -m 7 =m 

matrix consisting of the last k rows of and the subscript r+1 

in the Bayes risk refers to the number of observations for each parameter 

value t We now take as our standard R (6 ) = R (G^)« and seek a 

k.r — n r n 7 



procedure cp such that 



F) lim i sup 
n-» 00 1 Q 



i I E[(^(x ) - X lei )f] . ^ (i n )]}<0 

1=1 ^ J J 



We observe that R. ,_ ( 0 ) is not a desirable standard since if 3 is 

K,r+1 -n 

the class of binomial densities,, for example, then, as mentioned earlier, 

R .-(©) could not be achieved* 
k,r-t-l — n* 

We observe that theorems l) and 2) are still valid in this case 
when r (^ n ) substituted for and property p) for the 

property of uniformly asymptotically optimal of k th order . 

We define: 



A k - 



I x 1,1' ••• ’ x l,r ' 



^ X k,l' *•* ' *k,r 



where x is an arbitrary real number. 
s,t 
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k k k 

q = 1, 2 , . , m(A ) to be the m(A ) distinct matrices 

k 

obtained from A by independently permuting the elements 

k k 

within each row. Clearly 1 < m(A ) < (rl) . 



x k = 



X J-k+l,l X j-k+l,2 
X j-k+2,l X j-k+2,2 



.X 






X 



j,2 







M.(A k ) 

J 



t 1c 

' 1 if there exists q = 1, ... , m(A ) such that 

x k = A £ * 0 = k, k + 1, . . . 

=J,r (q) 



= < 



0 otherwise 



A k ) = M J (A k )h(X^ r+1 ) j = k, k + 1, ... 
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P 1 (A k ) 



£ M (A k ) 

....1^ J 

in(A k )(i - k + 1) 



Pi(A k ) 



E Zj(A k ) 
Jf* J 



mi 



(A K )(i - k + 1) 



P ± (A k ) 



P^) 



P*(A k ) H 



if P ± (A k ) > 0 



elsewhere 



r+1 

—XT 7 h(X ) if i 
r 1 s=l 1,S 



<£(&) = / 



- 1, . , k - 1 



- B 



, 1/2 






B' 



1/2 



if P*' 



B 



1/2 



) < 

i “M* ~ 

i. k^ k 1^ • • o 

if -B 1//2 < PflX^ ) < B 
.1 =i>r 

i - k, k + 1 ^ <i • • 

if B 1//2 < P*^ ) 

— i x =n.,r 



k k k 

We shall prove cp = (9^; Vq’ ‘ ’ ‘ ^ ^ as P ro P er ' ,: y P) provided the 
following condition on 3 is satisfied. V e 0 < 6 < 1 
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lira - T P 
n . , 
n-> 00 i=k 



“ i k r 

i n n 



• v f l V k+ , (X l-W,s> < iE 
j=k i=l s=l j-k+i 



- 0 uniformly in 0 , 



where X. . has probability mass function (•)• Such a 

i-K +£ ’ 8 0 i-k+i 



condition is satisfied, for example, if 0e ft xe2 =» p^(x) > t)(x) > 0, 
since in this case 



. J-, *6. , , /‘"'i-k+i 

L j=l i=l s=l j-k+i 



Pa (X- m) < i £ 



= £•••£* 

x.e2 x, e2 
1 kr 



E ft ft 



L . , , , , P 0 . , «^ X (i-l)r+s^ i 

L j=l i=l s=l j-k+i 



II fi P 0 (x (i-l)r+s ) 

i=l s=l i-k+i ^ X)T S 



< [ *•* I 1 



x^e2f 



v' 



ft ft K*, 



) -A-l 

■«. s=i ’ i 1 -^ 



ft ft 



P fl ( X 

i=l s=l i-k+i 



0_. /"(i-l)r+s > 



where, as before, l(a, b) = { 



1 if a < b 



0 otherwise , 



But V & > 0 there exists a set c 2 such that 21 has only a 



finite number Ng of elements and P[xe2g] > 1 - Hence: 
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z ••• z 1 

x n e2 x. e9 ", 

1 kr 



ft ft 






1-1 S=1 ' ji-6 



A A 



1=1 s-1 % i 



5 I • • • Z 1 

X l e9: 5 x kr £ "8 



ft ft 



.), 



M=1 s=l < i - 1 > r+s i 1 - 6 



+ 1 - (1 - 8 ) 



kr 



< 1 - (l - 6)^ r for i > where — — — ■ ■ < min [t](x)] 

( i ) e xe£~ 
o o 



— o 



Since 8 was arbitrary the result quickly follows * This is not the 
only case, however, in which the condition is satisfied, as was seen in 
Sections A and C. 

The proof that Cf) has property F) follows the same general lines 
as in our other examples. V i = k, k + 1, ... 



Let: Q.(A ) = ^ u IL * Q 

1 1 k 1 j=k i=l S=1 ^j-k+i £ ’ S 



X ft ft 



Pg ( x * J 



Q|(A ) - i _ k L, L, 

j=k xe£ j £=1 s=l “j-k+i 



Z Z h(x)p fl (x) 



ft ft 



p e . .., (x t, s ) 



R. be a set of k X r matrices such that P[X. eR.] = 1 
1 =i, r l 

and A k eR i Q(A k ) > 0 



89 



Then for A eR. we have that one version of 
1 

. k k Q*U k ) 

E[X(0 )|XV =A] is equal to if. (A ) = — - — rr- when 0 has 
m — 1 , r 1 , r 



Qi(A k ) 



m 



probability measure 



But 



E[M j ( ^ r^^i r = ^ = A A P 0 ( x 

J ~ X > T _1 ^ r n-1 0=1 r+P. 



P(Q \^p Q 
q=l i=l s=l j-k+i ' 



) j = k, . . . , i - k 



and 



EtZ J^,r>l5?,r " ^ - E[M J (A k )lE[h(X J>r+1 )] 



k r 

= m(A k ) I] U P© s ) Z h (x)p 0 (x) 



i= 1 s=l j-k+i ' xea: 



j — k, o9o , i - k « 



Hence, arguing as in Section A, 



P[ l P i (X i,r ) " >5 i (Ak)| ^,r = ^ ^ 2ke 1 



45. -2 ■ i -, k — 5 2 

k i 



k 



provided S . > ; — ; — 7 

* 1-1-k + l 



Since Z. is not bounded we are unable to use lemma 3) • We can, however, 
J 

use a simple Chebyshev bound, observing that Var[Z.(A )] 

J 

2 

< E[h (Xj t+ 2) 1 — an d hence using an argument similar to that in the 

proof of lemma 3) we haves 
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- Q i (X lr>' >8 i( Ak )l^,r = Ak ] 



< 



k 2 B 



(i-k + 1 )|5 1 - rr f ri |‘ 



provided 5 > 



i - i - k + 1 



It thus follows, using a modified version of lemma 4) , that : V A eR 



, (5, + B 1j ^ 2 5 . ) , 

P flh(5l> - ^ (A k } l£, r = V 



45 _2 

. ov 1 k i . 

< 2ke e + 



k 2 B 



(1 - k + 1)(S ± - t _ * + p 



provided 5 > 



i - i - k + 1 



Thus we may take 



5. + B 1 ^ 2 6 . 
l i 



«i 



if 6 . > - 



i — i - k + 1 






otherwise 



2k k&1 ~ 2 ^ ^ - 



k 2 B(i - k + i) 



[(i - k + l)5 - k] 



2 if 5 i>r 



k + 1 






otherwise 
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Let 




“i 106 1 lf 0 > k i lA 

± lA if Q i - (i - k + l) log i 



5 i=< 



0 



otherwise 



Then 



clearly lim — £ + > a^] = 0 uniformly in 0. 

- T P[Q. < a ] 
n . Lj . i i 



n-» °° i=k 



Since our assumed condition assures that lim 

n-» °° 



= 0 



uniformly in 9, theorem 2) is satisfied, and <£ has 
property F) . 

We observe that in this case, as in the binomial., the choice of which 



information to neglect at the i 



th 



stage was arbitrary. In particular 
k 



r could have been defined in any one of (r + l) 1 '" ways. We thus 
could have defined (r + l)^ essentially different estimators cp^ u , 

each of which would have the desired properties. As in the binomial case 

(r+l) k 
He 



it may be shown that cp^ = 



T-, 



K ^ , j_ u is an improved estimate, 

(r + 1) U =1 * 
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We note that if 3 had been defined as a class of absolutely 

continuous distribution functions* a similar decision procedure could 

have been derived „ As in the normal and gamma examples* a sequence 

(c.) would allow us to treat this continuous case as we did the discrete 
1 

case* using lemma 5) to show the appropriate limits hold* 

Go The empirical Bayes problem* 



We now consider a modification of our decision problem in which the 
sequence {0^} is not an arbitrary sequence* but is instead a sequence 
of observations of random variables* If these random variables are 
independent and identically distributed then the problem has been called 
the empirical Bayes problem* Many fine articles have been written on 
this problem and the results obtained have inspired this paper* We 
shall here* however* consider a more general form of the problem* 



Instead of assuming the 0^ to be independent observations of a random 

variable 0* we assume the sequence {0^} to be a realization of a 

stochastic process {©^: i = 1* 2* * * * } which is strictly stationary 

of order k* In other words for any k positive integers 

i^* i^* o.o * i^ and any positive integer j the k dimensional 

random vectors (©. *0. * .»• ,0. ) and (©. ,.* 0. „** * 0, . ) 

12 k 1 J 2 d k d 

are identically distributed. In particular, we suppose that 



V i = k, k + 1, 



the vector (0. , , ... . 

i-k+1 



0.) has distribution 



function Gp^) . Thus if G k (y k ) = ][ G (yp for some G we would 
have the standard empirical Bayes case 
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If G were known and if 0^ were distributed independently of 

(0,, 0 , ... , 0. ) then the standard Bayes argument would yield 

-L c. I" K. 

A = E[0 |xj] as an estimate for 0 ^ which minimizes the expected loss 

and achieves the Bayes risk R(G^) • Even if 0^ were not distributed 

independently of (0 , 0 , ... , 0 ) A might still be a ,T good ,f 

1 2 l-K. 

estimate, and the risk R(G ) a reasonable risk to attain* We shall 

"t h 

show that any procedure which is asymptotically optimal of k order 
(derived under the assumption of an arbitrary 0) will also achieve 
asymptotically an average risk less than or equal to R(G^). To be more 
precise, we shall show the following: 

Let: 0 be a bounded interval of the real line. 

{©^, i = 1, 2, . . . } be a strictly stationary stochastic process 

of order k. 

G n be the joint distribution function of (©^, , 0^) 

n = 1 , 2, » . . 

& be the class of all possible sequences of distribution functions 
(G n ; n = 1, 2, . • • } such that G n is the n dimensional 
marginal distribution obtained from G n+ ^, G n satisfies the 
above definitions, and G n puts probability one on for 



9 ^ 



all n. 



R(cp^, G 1 ) be the risk of using the estimate CfL for when 

the vector 0 is distributed according to G 1 . 

This risk depends, of course, on the class 
3 = (p 0 ( • ) : 0eft) . 

^ n 

R(9 , G ) = - £ R(cp . , G x ) where G 1 is the i dimensional 

i=l 

marginal distribution obtained from 
G n . 

We now state and prove a generalization of a theorem by Samuel [ll]« 
Theorem k) Let 3 = {p (*) : } be a class of distribution functions. 

C7 

k 

Let ^ he a decision procedure which is asymptotically optimal of 
order for 2s . Then 

Tim R(q>jj, G n ) < R(G k ) 
n -> 00 



n k 

for all {G Jejfo If cp is uniformly asymptotically optimal then the 
above inequality becomes 



lim 
n -> 00 



sup [R(3> k , 

{G n }e£ 



G n ) 



- R( G ) ] r < 0 
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Proof : 



Let R(qr, 9 ± ) = E[(q£(x£) - 0.) 2 ] . Then: 



R( V ° n ) = ^ I R (V G±) = n £ - ©i) 2 l 

i=l 1 i=l 

= i I E(E[ (cp^(x i ) - e.) 2 ) I©.]) 
i=l 

-i 1 E[H(<pJ, 9 )] . 

1 = 1 



For the remainder of the proof we shall let E^[ e ] represent expectation 

where V i > k ••• , ©^ have a priori distribution function 

k 

G , and E^[*] represent expectation where ©^ ^,+2/ * ce } ®± ^ave 

k t h 

a priori distribution function G^, the k order empirical distri- 
bution function generated by 9 9^, . .. , 0^. We now let A(X^) be 
a form of E [0. |X^] and t(X^) he a form of E [0 |x^] . Then A 

X J i 1 i~i 



2 y-y 



achieves the risk R(G ) and the risk R^(£ n ) • We observe V 



E i [(A( 2j) - e/ |e o = ¥< A <$ 



e / |9 j = 2 r ] 



We call this common value L(0 ) . Then 
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V- n > = ¥<*<i> - V' 1 

< E 2 L(A(X k ) - © k ) 2 ] 

= E 2 {E 2 [(A(x k ) - e k ) 2 |© k li 

= n - k + 1 X L - e i-k+l^ 
i=k 



8 i> 



Hence : 



e i [ W ! 5 n _^! i E i[ L < e i.k + i' - e i )! 

i=k 



ttttt I - R <° k > 

i=k 



But since cp k is asymptotically optimal we have 



E -j lim - V R(cp k , ©.) - R. ( © ) f" < 0 

1 I n i -i k -n I - 

^ n—» co i=| ^ 



n-> co i-l 



and hence,, since our losses are bounded, 



lim 
n-> 00 



i l E i [R( ^' 

i=l 



< 0 



=? lim 
n-» oo 



R(ce k , G n ) - R(G k ) 



< 0 . 



The proof of the second part of the theorem follows immediately, 

Q.E.D. 
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Corollary, 



If in addition to the assumptions of theorem k) we add the condition 

that V j = k + 1, k + 2, . . . 0. is distributed independently of the 

J 

vector 0. , then the two conclusions of the theorem may be replaced 

— J “K 

b y 

lim R(c£, G n ) = R(G k ) 
n-» oo 

and 

lim R(cp k ^ G n ) = R(G k ) uniformly for {G }e J& 
n-> °o 



respectively , 

Proof : 

To prove both parts of the corollary it is sufficient to show 

lim R(cp k , G n ) >R(G k ). But since 0. is independent of 0. , R(G k ) 

' n j .1 ”K 

n-» 00 

is the minimum risk that can be attained by any estimate of 0., Hence 

J 

R(cp k , G^) > R(G k ) for all i > k„ It may be shown that 

i<k=^>R(G 1 ) > R(G k ) so that R(cp k , G 1 ) > RCg 1 ) > R(G k ) for all 

i < k. Thus R(c£> n , G n ) > R(G k ) for all n so that 

lim R(cp, G n ) > R(G k ) as desired. 
n-> oo 

Q o E » D o 
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